How to get feedback from more people in less time

According to usability testing guru Steve Krug, “usability tests are about watching one person at a time try to use something (whether it’s a website, a prototype, or some sketches of a new design) to do typical tasks so you can detect and fix the things that confuse or frustrate them” (Don’t Make Me Think, 3rd edition: p.113).

Let's call this kind of testing “traditional usability testing”. If you are not familiar with it, I recommend that you watch this demo usability test by Krug to get an idea of how to run such a session. In a nutshell, traditional usability testing:

• Is a one-to-one session, one participant at a time, giving you feedback about your product.
• Typically takes about 60 to 90 minutes per participant.
• You get feedback from 3 to 10 participants over a couple of days.

Group usability testing

If you ask for feedback early and often, traditional usability testing works great. But what if you have the opportunity to get feedback from several participants and only have up to 90 minutes with the whole group? Welcome to the world of group usability testing (or “usability testing on steroids” as I ended up calling it affectionately).

I faced this challenge as the Lead User Experience (UX) Designer of the Open Targets Platform when my team and I were attending project review days, site visits and conferences. During these events we had the opportunity to ask 5 to 10 target users for feedback on new or existing features. We would typically have 60 to 90 minutes with the whole group to get their feedback so traditional usability testing was not feasible within this timeframe.

So together with the developers and the senior stakeholders of the Open Targets team, we developed the group usability testing method. The main idea is to run several usability tests in parallel. I pair each participant with a facilitator from our team. The facilitator can be a developer, a support or outreach officer or a more senior stakeholder such as a project manager or a team leader. Everyone is in the same room.

I act as the master of ceremonies: I welcome the participants and introduce the session to put everyone at ease. Then, I introduce the task and ask the participants to think aloud and give us honest feedback (I follow Steve Krug’s script during these steps).

The participants are then trying to carry out the task and telling the facilitator next to them what they are thinking and trying to do. The facilitators are taking notes of their feedback on post-its.

A group usability testing session: Participants are paired with a facilitator and carry out a task while they are thinking aloud. The facilitators are taking notes of their feedback on post-its.

Once the participants are done with the task, I start gathering their feedback. Usually we would have printed screenshots of the pages that the participants are meant to look at and put them on the wall (or on a foam board). The facilitator reads out the feedback on each post-it, the participant provides clarifications if needed and I place the post-it on the corresponding screenshot. In that way, the participants are exposed to each other’s feedback and can comment on it.

Gathering feedback during a group usability testing session: The facilitator reads out the feedback on each post-it while participants comment on it. The post-its are then placed on the corresponding screenshot on the wall.

Group usability testing generates a lot of feedback. Discussing their feedback with the participants as we are collecting it helps us identify commonalities and organise it into themes. We try to have a first pass organising the post-its during the session. Soon after the session we have a second look to prioritise and transcribe the most important issues and define follow-up actions.

Transcribed feedback from a group usability testing session with follow-up actions.

Group usability testing fosters team collaboration and rapport with users

I was initially very sceptical about this way of getting feedback from users but it turned out to be a great way for our team to collaborate before, during and after the session. It gives the whole team the chance to talk directly to more users than traditional usability testing in less time. It has the added advantage that users are also exposed to each other’s views so we can spot common issues versus individual preferences more easily.

Yet the most unexpected outcome was that the participants loved it and kept asking for more so much so that some of them later became our facilitators. This gave us the opportunity to involve them in other activities such as user research and design workshops and build a community of UX enthusiasts among our target users.

Building rapport with the scientists who participated in group usability testing was viewed favourably by the senior management of the project too (some of whom were involved in the actual sessions). So group usability testing became an integral part of the Open Targets quarterly meetings and helped turn our project leaders into UX advocates.

Group usability testing top tips

We tried group usability testing with varied numbers of people and time periods: 5 to 10 participants within 60 to 90 minutes worked best. Less than 60 minutes is not enough time and after 90 minutes participants get tired. Larger groups are harder to manage. If you have more participants (and more time) it’s better to run more than one group testing session instead. Otherwise run just one session with up to 10 people (Believe me you’ll get enough feedback for your next couple of sprints).

We run the sessions during events which are attended by our target users and collaborate with the event organisers to recruit participants. Most of our participants are familiar with Open Targets (but not necessarily with the actual feature that we are testing) so we can keep the introduction brief and to the point and spend most of the available time on collecting and discussing their feedback.

Because my colleagues had attended traditional usability tests before they can help their participant think aloud without making them anxious or leading them. I found that most people in my team can facilitate in a group testing session after they had attended their first two or three traditional usability tests as facilitators or observers.

Krug contrasts traditional usability testing with focus groups, in which a group of people are asked for their opinions about a product. One of the problems with focus groups is that some participants may try to dominate the discussion, which can prevent others from expressing their views and result in groupthink. As in traditional usability testing, we ask our participants to try out the product rather than just talk about it. The initial pairing helps us capture and discuss individual feedback and avoid groupthink.

Of course, group usability testing is not the only method in our toolkit. To design the Open Targets Platform we combined it with user research to better understand people’s motivations and more traditional usability testing to get more in-depth feedback within an iterative and lean approach to UX design (see Karamanis et al 2018 for more details).

Doing a couple of days work within a couple of hours

In conclusion, group usability testing enabled us to:

• Do a couple of days work within a couple of hours.
• Leverage user feedback without compromising on its quality.

Through the UX for Life Sciences (UXLS) initiative, I came across other colleagues who needed to get feedback from several users in a similar setting. This motivated me to run a workshop at UX Cambridge 2018 on group usability testing. The workshop was fully attended and the participants were very engaged so I’ve written this article to encourage more product managers, UX designers and front end developers to try it out.

Participants of my workshop in UX Cambridge 2018 practicing group usability testing.

I am grateful to the participants of our group usability testing sessions and my colleagues and management at Open Targets and EMBL-EBI who gave me the opportunity to try out this method and get better at it by sharing their feedback.

If you run a group usability testing session, please let me know how it went!

Related links

• Demo usability test by Steve Krug (25 minute YouTube video)
• Usability test script by Steve Krug
• Michael Margolis tests a prototype with a customer (8 minute YouTube video)
• Slides of my workshop on group usability testing in UX Cambridge 2018
• How we designed the Open Targets platform: Karamanis N. et al. (2018) Designing an intuitive web application for drug discovery scientists, Drug Discovery Today Volume 23, Issue 6, Pages 1169-1174.