A little while back we invited you to ask anything you ever wanted to know about interviewing users. And you did. We gathered a bunch of very interesting questions and a lot of food for thoughts for all of us. Thank you all for that! Also, many thanks to Steve Portigal, who did not only contribute a truly inspiring book to the UX community, but who also shared his ideas and insights with us by answering all of your questions.
We also promised you to give away three copies of Steve’s book Interviewing users for the three most interesting questions. Together with Steve, we selected our three winners. Congratulations to Dennis, Elise, and David!
Here are their questions:
Elise: What are your best tips for handling low-energy / quiet interviewees? I wonder if my extroverted body language is making them shut down more! Also, I have R&D team members who love to be involved in customer-facing activities. What’s worked for you in training teammates in user interviewing?
Steve: See my handout for teammates for that question. In Chapter 8 I talk about “When the participant is reticent” – if you are extroverted maybe you are expecting or hoping that they will come off as extroverts. Maybe everything is just fine and that’s the way they are?
Dennis: Talking to users often provides insights that are valuable to people outside of ux/web teams. What advice do you have for integrating user interviewing into other areas of a business so those insights are shared?
Steve: Try to bring those other groups in as stakeholders at the beginning (and along the way). Can you create deliverables – or meetings where results are discussed – for those other groups?
David: How hard is it for your friends and family to be unbiased interviewees? Is this considered unethical?
Steve: There’s a lot of issues here; I think opportunism is an important aspect of participant recruiting. Ask yourself “who can you get?” At the same time, it’s crucial to identify what are the characteristics of your participants that will make your research valuable. So you need to balance those two.
The problem with friends and family is that they are actually harder to interview. Imagine you are interested in people’s vacation planning process. You can ask a stranger “What was the last vacation you took?” and it’s so much easier to ask that question authentically because you really don’t know. If you are interviewing a family member, you know what their last vacation was. Sure, you can say “Now I think your last vacation was to Italy, is that right?” But already the dynamic has shifted and you are sort of now both playing the role of recapping for the camera things you’ve already talked about. It’s just so much easier to deal with a relative stranger.
My compromise is to get friends-of-friends. So you get the expediency of working your social network but you meet someone you know a lot less about.
I’m not sure what the ethical issues are here; to me it’s more about what will get you the best data. I certainly wouldn’t suggest you misrepresent your sample to someone else though.
Not in the top three, but still very inspiring
Gavin: How do you best eliminate candidate selection bias, whether or not you are involved in finding users? Right now, I am gearing up for user testing and am a bit nervous about a certain client selecting the users to be interviewed.
Steve: Seems like you need to find a way to have the difficult conversation with the client about the concerns you have about getting them the best information that can help them the most. You obviously know your stuff and that’s what they’ve entrusted you to bring, so step up. Note: this is much easier for someone to write in a comment box than it is to actually go out and do it! But consider this a vote of confidence, amirite?
Dante May: Slightly off-topic, but what advice / lessons learned can you give from “Interviewing Users” that can be applied to interviewing internal and client stakeholders?
Steve: The pieces in the book that talk about rapport, listening, and asking questions (Chapter 2 and 6) apply nicely to any interview setting, I think.
Edje: How do you manage giving all users equal opportunity to give honest feedback and not allow discussion to be dominated by tangential aspects as some people desire to impress or show off to others in a participative feedback session?
Steve: It sounds like you are describing a focus group setting and the behavior you are describing is well known by social scientists. If you don’t want to have people socially performing with each other, don’t bring them into a group. I talk in Chapter 8 about interviewing multiple participants, but in that case I am thinking friends, family, not strangers thrown together suddenly. Why do that to them (or yourself)?
Frank: Some users tend to steer their answers towards telling you what specific functionality they want from the ultimate product, no matter what you ask. How can an effective interview be conducted in such cases?
Steve: This is about how the interview is framed – from the point you make the initial request, to the introduction. However, in many cases, people have stuff to tell you and you need to let them tell you that before you go forward from there. You also can keep asking why and why and why for every feature request they make. You can ask who else has the same need, etc. You want to get the discussion to needs, but you can let them dictate the starting point if they are so passionate about that.
Ian: What are your techniques to make sure that questions are completely non-leading and non-subjective? How do you keep users on track? How far do you let them go before you steer them back? How much do you prep users before their interview? How can this influence their behaviours and responses? Looking forward to reading the book!
Steve: I will give people some context for why we’re talking to them, but most people understand “market research.” I definitely do not go on and on about alphas and rollouts and BUs and other things they don’t understand or care about. I say just enough and at the end I ask them if they have any questions for me. Some people will say “What will you do with this information?” or “How many other people are you talking to?” – but the key is to orient this discussion – at least until the end – to their world, not yours.
Erik: How do you balance interviews from different groups of users. I work for a large membership organization. We certainly want our site to give our members them what they want, but we also want to influence non-members. There are also small groups of users that heavily influence our website decisions but are unlikely to subject themselves to interviews in any representative numbers: I’m thinking specifically of journalists and policy makers.
Steve: One reason to look at different user types is to highlight contrasts you wouldn’t otherwise see. The other thing to keep in mind is that there’s a difference between who you do research with and who you design for. The first gives you insight about how to do the second.
Kate: How do you handle a participant who seems determined to find fault with any website or product you mention? Do you treat their feedback as valid or take it with a large pinch of salt?
Steve: It’s always helpful when getting feedback to understand “why” – quant methods are better for simply tabulating the number of issues out there, but you have the chance to dig in. The why may be the same or different and may be enlightening in different ways. And yes, I think that we bring our own interpretation to what we’re hearing from people. If someone says “I hate life and I hate people” and then says “I don’t like the way this peanut butter tastes” then we’ve got some good context for that feedback. Thank goodness we’ve had the opportunity to learn more deeply how they look at the world!
Marc: How can you prevent users telling you what they think you want to hear, rather than how they actually feel?
Steve: You are describing the need for rapport (see page 20 of my book) – it’s up to you to establish this dynamic with your participant. You also need to hear the difference between those two types of responses and adjust your questioning or follow-up further as necessary.
Ann: How can I avoid leading questions when others write the questions and push for them to be asked? How do you avoid or redirect the interviewee answering hypothetically for their demographic? How do you get better responses when the interviewee doesn’t seem engaged?
Steve: You shouldn’t be asking the questions as written. The interview guide is just that – a guide. Otherwise, you are just moderating a survey and not conducting an interview. Chapter 6 of my books talks a lot about how to ask questions and might be helpful for you.
Annemarie: How can you beforehand make sure that you acquire participants that are ‘good’ at thinking aloud during performing a certain task.
Steve: There’s a standard articulation question that should be in your screener (sample.
We often do a 5-minute call before we meet for an interview to develop rapport, see if they have any questions, verify some of the things we are asking about, etc. If you have an exercise for them to try, then you could do it then, too. I think if you decide not to go forward with them after that, though, you should give them a small incentive as thanks.
