Top tips for usability research interviews

What are usability research interviews?

Usability interviews are a common method for identifying whether a prototype web/app design is likely to be effective if released, or for finding out the issues and unmet needs of live sites. Participants will be asked to complete tasks on the prototype or site whilst being observed and asked questions.

In this short article I’m going to imagine that we’re talking about a usability interview for a travel booking site so I can use some specific examples. You’ll need to generalise the principles to other contexts.

I’m not going to cover the setup — the interview could be in-person or online, and you obviously need to ensure you’ve recruited the right people to take part.

1 — Be clear about objectives and write a discussion guide

The objectives of a usability interview might seem obvious but if you have a number of stakeholders there can be many unspoken assumptions about what you’re trying to achieve. For our travel site one stakeholder might be thinking that the key objective is about figuring out how travellers want to book flights and hotels together — which do they want to do first? A second stakeholder might have a particular interest in how travellers trade off price and standard of service, whilst a third is more focused on perceptions of the brand.

These interests are not necessarily mutually exclusive, but it can avoid a world of pain if you get all the objectives written down and try to get agreement on priorities. Whether you get that agreement or not you need to go into the interviews with a sense of what matters, or you’ll end up with a wandering semi-coherent conversation and potentially not get satisfying answers to any of the questions.

2 — Structure the course of the discussion as a funnel

In the interview start off by talking about travel in general, whether the participant travels for work or leisure, alone or with colleagues or family, whether they go on packages or craft their own trip, and whether they travel near to home or further away. All of this gives a context for what follows.

Having got the big picture focus in on where they’ve been recently and plan to go soon, how they planned their journey and which sites they have used as well as other sources of information. Get some headlines on the process they follow and what makes a good site and what frustrations they encounter.

Then you can introduce your own brand with ‘have you ever used the brand x site?’ if they haven’t already mentioned it and understand their general view of it before diving into the tasks.

3 — Be flexible with the discussion guide

The guide is exactly that — a guide, not a script. The interview will go better if it feels more like a conversation than an interrogation and so, whilst you want to make sure you cover the topics, you should still be responding to what the participant is saying, and listening out for nuggets of useful information to take away and that might influence the direction of the conversation.

For example, if at the start of the interview the participant says they want to go to Jamaica one day and that they dislike New York, then when it comes to asking them to do a task ask them to plan a trip to Jamaica rather than New York even if New York had been the original plan. It makes it more real for them and you’ll get a better sense of what they are likely to do in real life.

Of course, if you only have a prototype that allows for New York trips there’s not much you can do about it, or if there are unique features to booking New York that you want to investigate, but the more flexible your prototype can be, the better.

4 — Stay on track

Whilst you do want to be flexible in your approach you also need to stay on track and make sure you address the research priorities. The participant though won’t have the same focus. Some people will try to be helpful and focus on your questions.

One problematic type though is the person who constantly tries to anticipate your questions and give you answers that they think you want. With these you may have to pause and gently remind them that you are a neutral player and what would be most helpful would be knowing what they genuinely think.

Another problem type is the person who sees you as a representative of the organisation sponsoring the interview and takes the opportunity to vent about any issues that you’ve had. In such cases I’ve give them a small amount of time to let off steam, and then remind them that whilst you appreciate their perspective you do need to get through some questions in the time you have, and that you’ll promise to pass on the comments to the organisation. And make sure you follow through on the promise.

5 — Ask open-ended questions

When the participant finishes a task it’s easy to ask questions such as ‘so was that easy to do?’ or ‘did you like that’ but these are leading questions that set a context for the participant. They are being asked to reply in terms of ‘ease’ or ‘liking’. Bear in mind as well that you’ve been observing the behaviour, so you know if they struggled, or spontaneously commented on how easy something was.

It’s better to ask a question such as ‘so how did you find that?’ or ‘What are your thoughts about that’? It leaves the way open for a wider potential range of answers.

6 — Don’t suggest answers unless you have to

Less experienced interviewers are more likely to make this mistake as they often find it harder to ask short questions.

For example, you might ask ‘What do you do when you first plan a trip? Would you start on the web on your laptop, or use your phone, or would you talk to people?’ Such a way of asking potentially constrains the answers that might come back. Instead, just ask ‘So what do you do when you first plan a trip?’ and leave it there. There are likely to be times when the participant doesn’t understand the question or mis-interprets it, in which case you should try to re-phrase the question but still with an open question. Sometimes though it is necessary to give examples of answers, so you get an ‘Oh I see what you mean response’.

7 — Be a good listener

It’s easy in an interview to be distracted by thinking about what to ask next, or what’s going on outside the window. Remove as many distractions as possible from the environment for both of you, and good preparation will help to cut down on working things out as you go along.

You don’t want to miss the one key item of information because you’re thinking about what to have for dinner.

8 — Be comfortable with silence

Often when you ask a question the participant will sit and think. Let them. Don’t feel the need to jump in and fill the gap — if there’s an issue with the question let them say so. Even after someone has answered a question there may be more to come to don’t always start talking again as soon as they finish. This can be something of a judgement call so watch their body language and face. If they look like they’re still thinking about the answer they’ve just given, then give them the space — the next thing they say might be a gem that you’d have lost had you moved the conversation on.

9 — Don’t take notes

This might seem odd at first, but if the interviewer has to take notes during the session, then they’re not giving their full attention to the process.

You should be recording the sessions. If you’re using something like Zoom or Teams to do the interview, then these will make an automatic transcript if it’s turned on which will be good enough. Even if you’re in-person you can still use the computer to record.

There are commercial platforms that will do the recording and transcript and also provide additional tools for getting video clips and creating highlight reels. There are also tools that will integrate with the likes of Zoom to provide some of these additional capabilities.

10 — Keep your ego out of it

It’s possible that during the interview the participant may not understand things that you think are obvious. Put bluntly, you might end up thinking that they are stupid. You can’t respond to that other than adjusting your approach to meet their needs. Ask simpler questions or give more context.

It’s also possible that you’ll uncover opinions and points of view that you strongly disagree with. Don’t be tempted to get into debate, that’s not what you’re there for. Your job is to find out about how this person thinks and behaves — not to try to change them.

Bonus tip — Don’t spend time explaining the system

When interviewing someone about an interface, and the interviewee doesn’t understand the system or gets it ‘wrong’, they may ask if they are doing it right, or how it works, or what they should do. This is not (usually) a cue to explain. You could respond ‘so how do you think it should work’, but the task is to hear from them in the time available. When the interview is over you can explain if you both want.

Conclusion

Clearly there’s more to it than just these ten points — and each person with experience might have their own different list, but I’m confident that there’ll be a lot of overlap and agreement. Each point could be expanded into greater detail so if any are new or thought-provoking it’s worth digging deeper to get other perspectives.

As people do more interviews, they will usually refine their approach and become more effective, learning deeper insights. If you’re new to it and can apply the principles here, then you’ll be on the right footing to make a good start.