2013_08_15_structured_research Theory

Step By Step Guide To More Structured User Research

User research. A concept that has long found acceptance in most companies. We know that user centered design is important – and there are plenty of different job positions that ensure the user is included in the product development process.

Be it during the initial conceptual phase, or after a product is already for sale – the user decides whether a product is fun, useful, and easy to use – in other words whether or not it will be successful.

User research is essential for building successful products. But what does user research really mean? And how do you do user research? Where do you even start?

This step by step guide will not only show you where to start, but also help you to conduct more structured user research:

1. Define specific research questions

First of all – even before you know who to test, or what method to use – you need to know WHAT you want to test. Your research questions are the base for any structured research. Not only do they justify your research, they also help you to stay focussed throughout the entire research process.

Research questions should be specific. Questions like “Is our product easy to use?” or “Do we need to make any changes before we go live?” are not very good questions.

Try to be as specific as possible and focus on critical points, or new concepts. So instead of looking at the entire product, ask “Is this one feature easy to use for existing users?” or “Do new users know where to find this feature?”, “Do they know how to use it?”, etc.

Keep in mind that your research questions define the focus of your research. If you find yourself answering different questions along the way, it’s time to pause and reflect. Is it a relevant question that you missed to define at the beginning? Then add it to your research. Is is something fun or interesting, but not really relevant at this point? Make a note for later and move on.

2. Define your target audience

Next, define your target audience. This does not mean define the target group for your product. Rather, you should focus on those people relevant for the research at hand.

Sure, sometimes, this can be just any random selection of your users. However, it can also be a specific group, such as heavy users, long-term users, or new users. Or you might need to test potential users – depending on the level of experience and interest you had in mind when designing your product or feature.

Your target audience should become clear once you have defined your research questions. When defining critical points that you want to test, always define for whom you expect it to be critical. Those are the people you should test.

3. Select a research method

So now you know why you want to conduct research, what you want to test, and who you want to test with. Next, you need to decide which research method best suits your research.

Considering the vast amount of research methods out there, this can be a challenge. Besides taking into account your financial resources, you should ask yourself the following questions: Do I want to conduct remote or in-person research? Do I want quantitative or qualitative insights? Can I use an automated tool or do I need to work with moderated sessions?

Obviously, this depends on the phase of the design process you are in. In early stages, automated remote testing can be sufficient to gather feedback and learn about preferences. Also, specific details can easily be tested remotely. If you are planning on a more complex research with focus on qualitative insights, you might be better off doing an in-person session.

Check out the UCD toolbox to help you find the right method for your next project.

4. Design your research

Once you have decided which method will work best for you, it’s time to design your research in detail. This means you need to think through your research question for question.

Again, your research questions are critical. Here it is important to realize that your research questions are not identical with your test questions. Good user research doesn’t mean you ask your users exactly what you want to know – but you know what to ask them to get the answers you need.

For example, your research question might be something like: “Do new users know where to find feature A?”. Then don’t prime them with ‘feature A’, but think about what it is that feature A does – such as effect B. Then ask them something like: “What would you do to trigger effect B?”, or “How important is effect B for you?”, or “When would you use effect B?”.

Your research design also includes all test materials, such as prototypes, mockups, etc. Make a list with everything you need to conduct your research.

5. Pilot session

Do you think you are ready? Did you design your test questions? Great, then you are ready for your pilot session.

A pilot session is a great way to validate your research. You can either use a user from your target audience, or just get a colleague to fill in. This might seem like wasted time, but it really helps to identify flaws in your research design.

You will see that a lot of times, you either need to adjust your test questions, or change their sequence, or you might need to add some extra questions to really get to the heart of a problem. Also, you might need to offer more background information, or less in a matter of fact.

6. Get organized

Once you are ready to get started with your official research, make sure you get organized. Meaning you should have a detailed time planning to give you and your team an exact overview of when to expect which results. Especially for projects with a tight time schedule, this is critical – and to be honest, in that case you should already have a planning since the beginning.

Getting organized also means to get everything set up for your research and to answer any remaining organizational questions. For example, you should know how many participants you need, and how you are going to recruit them. If you are planning on conducting in-person research, make sure you have a suitable location and any needed accessories at hand, such as a computer or camera. And of course, you need a prototype or interface to test.

You should also think about legal issues. What kind of data do you collect during your research? And what are you going to use that data for? Make sure you prepare any paperwork that you might need your participants to read, or sign.

7. Conduct research

Finally, it’s time for the fun part. Be it that you send out your online study to hundreds of users, whether you go out and interview people on the streets, or invite a selected group of people to a one on one testing sessions – now your research design is out to the test.

Make sure you have planned in enough time and and resources to assure reliable and consistent results. When conducting in-person research, this means you should do only as many sessions per day as you can handle without lack of focus. Also, if you have a whole team of researchers at your disposal, consider to leave the interviewing part to one or max two people. This way you create equal conditions for all tests.

8. Analyze results

Once you have gathered all your results, don’t wait too long before analyzing them. The more recent your impressions, the more detailed your documentation.

Now bring together your results, sort and summarize them. Depending on the method and tools used for your research, this report can look slightly different. But no matter its format, make sure you can easily groups findings and add comments. This is probably the most time consuming part, but it will help you to get a good overview of all your results.

Try to focus on negative feedback. While positive feedback is nice to have, it won’t be much help for improving your product. Once you have grouped similar findings, define the common issue they identify. Last but not least, you can add a severity rating for each issue and the estimated effort it takes to fix it. This will help you decide in which order to address those issues in the following design iteration.

9. Validate findings

Sometimes, user research reveals very critical issues. If that happens, there shouldn’t be much discussion. Just fix them. However, sometimes, issues can also be vague, or there might be the impression they don’t apply to many users.

For example, if you conduct in-person research and and one person identifies a seemingly critical issue, you might need more data to back it up. With an automated remote test, you can carefully examine a single issue and quickly find out if it’s scalable.
On the other hand, you might have conducted a large scale remote research, but now you are missing some details. In this case, a qualitative interview or in-person session might bring them into focus.

10. Define follow up research

Once you have a clear picture of all valid issues, go ahead and improve your product. However, this does not mean your research should stop here. Rather, user research should accompany your entire product development process.

User centered design is an iterative process in which you (1) analyze the context of use, then (2) specify requirements, (3) come up with a design solution, and (4) evaluate this solution with your users. If your design doesn’t satisfy the specified requirements, you go back and change it. Simple, right?

4 comments

  1. Jonathan

    I enjoyed your article Sabina, thank you. I want to ask if you have any tips or best practices regarding how best to get good responses to the questions. Is it multiple choice, do you always allow for answers like “other” or “not applicable”?

  2. Spook SEO

    Hello Sabina!

    You hit the nail on the head when you said that the research question shouldn’t be identical to the test questions. A lot of researchers make this mistake that’s why they end-up with weird results.

    Awesome post! Thanks for sharing!

  3. Sabina Idler

    Thanks for your kind comment. I noticed the same thing, that’s mainly what motivated me to write this post. :)

  4. Sabina Idler

    Hi Jonathan,

    I’m glad to hear that you likes the article. I think in general it’s important to be as specific as possible and (if you conduct in-person research) to keep asking until you are at the core of the issue. WIth remote research this is more difficult and the best thing way to make sure your questions grant good responses is to test them first. With multiple choice, it really depends on the question. Do you need people to choose one of your predefined categories? If not, or if there is a privacy issue involved, you should allow people to select “other” or “prefer not to answer”.

Leave a Reply

Your email address will not be published. Required fields are marked *