What do you do if your user testing and feedback process is not yielding valuable insights?

Imagine you're designing a new mobile app for tracking fitness goals. You've conducted user testing sessions, but the feedback you're getting seems vague and unhelpful. In this situation, you'd review your goals and methods. For instance, you might realize that your questions during testing were too broad, like simply asking users if they liked the app. Instead, you should have asked specific questions related to usability, such as how easy it was to log workouts or set new goals. By refining your approach based on these insights, you can conduct more focused testing that provides actionable feedback for improving your app.

Before you get a feedback, you should conduct a research. You research should be based on goals. For that there are various ways to do: you can get specific goals for your ideation or you can ask you team about the goals they may find useful for the research. For you methods, you should choose them based on the goals you've set and on the resources of your company. Most of the time, interviews and usability studies are considered as the best ways to get valuable insights on users because they will help you really identify the pain points of your users in a better way. Indeed, it is important to know earlier the goals and to know which methods to use for your research in order to get great data for your solutions.

Here are some steps you can take if your user testing and feedback process isn't yielding valuable insights: Revisit Your Goals and Methods: Are you asking the right questions? Are you testing with the right users? Are you using the appropriate methods? Analyze Your Data and Feedback More Deeply: Look beyond the surface Look for patterns Consider the context Validate and Prioritize Insights: Don't jump to conclusions Talk to your team Focus on actionable insights

If user testing and feedback aren't providing valuable insights, I reassess my goals and methods. I scrutinize the testing criteria, user demographics, and feedback channels. Adjusting protocols, refining questions, or diversifying participant pools can reveal overlooked insights. It's crucial to iterate and adapt until the process yields actionable data.

When user testing doesn't provide valuable insights, it's crucial to reassess your UX research framework. This involves ensuring that your user testing objectives are tightly aligned with your product's goals. It's also essential to verify that the questions posed during testing are designed to elicit actionable feedback, rather than just affirming assumptions. Additionally, employing a mix of qualitative and quantitative methods can uncover different dimensions of user experience, and reaching a diverse user group can prevent skewed results that don't represent your entire user base.

There are surely various ways to analyze your data and feedback. With your users, preparation is key before diving into this section. But let's think that you have conducted an interview. When the interview is finished, you can start creating empathy maps, personas, user stories, user journey maps, problem statements, hypothesis statements, value propositions, competitive audits, and more. You also need to analyze you data and feedback from your team by doing critique sessions, asking them one by one (depending on the company policy on getting feedback), and stakeholders interviews. All in all, you need to have great presentations to introduce your findings to your team. There you iterate with your data on your solutions.

Even when user feedback seems off-target, it still holds value. Dive into your test results, paying close attention to any user difficulties or elements they found straightforward yet unnecessary. Look for moments of hesitation. These insights can pinpoint where the actual issues lie, guiding you to refine your objectives and approaches for future tests. View it as a cycle where each phase informs the next, emphasizing continuous improvement.

North Star: Stay focused on your goal - it should be so ambitious that a solution for it brings so much value for both business and users. Prioritise ruthlessly only insights that drive you in the direction of your goal. Choose your goal wisely. If an insight is: - showing you the essence for the new solution - helping you build stronger new solution - identifying weak sides of the current state //consider it as it may be solved by the other insights above Anything else that could be an improvement, but don't bring much value and contributes on more clutter, cognitive load or friction for the user. Let it go. Adopt: LESS is MORE mindset.

Check and sort your findings to figure out what's most important. Make sure your insights are accurate by comparing them with real user behaviour or other data. Then, decide which ones need your attention first. This helps you focus on changes that will really make a difference, ensuring your updates are based on what users truly need.

Acting on insights is a critical step in UX that bridges the gap between research and design. It's essential to use these insights to drive iterative design improvements, ensuring that each iteration is informed by actual user feedback. This approach not only enhances the user experience but also aligns the product more closely with user needs and business goals. By continually testing and refining, UX professionals can create more effective and user-friendly designs.

After you prioritise insights identify 20% of them that will create 80% of the solution/value. Align your stakeholders with the goal, the test and the results. Few qualitative validation sessions /interviews or tests/ with open-ended questions with users that are neither involved in the development process nor biased by the current solution will give you so much insights, that you won't need much interviews, sessions or iterations afterwards. Stay open, listen and observe - behaviours, actions, motivations, frustrations.

Start making changes based on your sorted insights. Tackle the most critical issues first to make a big impact. Small, quick fixes can also boost user satisfaction early on. Use what you've learned to improve your product, and keep tracking how these changes affect user experience. This way, you’re always improving and staying responsive to your users' needs.

To help you and your team on the decision making process, you should first communicate in an effective way your insights. Keep in mind that this is one of the most important phases of your research. UX Designers are great communicators. During presentations you need to make a lot of explanations but the idea is that you need to use your data in every explanation you make. Then, you should use stories to impact more during your sharing and you should also use great visuals in order to captivate the attention of your people. This is the opportunity to show to your team members that you are someone who is approachable and respectful. Tell them clearly that you need to get their feedback in order to provide better solutions for the users.

Qualitative sessions will give you deep understanding, they bring very different insights from those that derived from usability sessions without think-out-loud protocol focused on measuring time-on-task, completion-rate, error-rate. Both are needed. However I would start first with qualitative approach until I create a solution of a decent quality, then proceed with quantitative sessions to refine performance. Communicate how an insight derived. Particularly in what kind of session. Let your audience build contextual intelligence by understanding the contextual conditions which created different insights.

Share your findings openly with your team and stakeholders. Use clear, concise reports and include user testing recordings to highlight key points. This transparency ensures everyone understands the user's perspective, making it easier to align on the next steps. Writing brief, focused UX ("Lean UX") reports keeps the information accessible and actionable, fostering a collaborative environment focused on user-centred improvements. As an example of lean reports, I often display findings with a traffic light system, marking the most common issues as Problematic (red), Good (green), and Bug/Potential issue (orange).

Designers are not the users. Many times we assume a solution will be easy for users, then it fails when tested. Learn and communicate how team's assumptions and expectations met reality of users. Learn about users behaviour, mindset, mental models and present the information this way on your team. Whatever you experienced during sessions, make it a learning experience - about the context, the users, their interaction and integration of the product/service in this system.

Additional Tips: Conduct follow-up interviews: Consider conducting follow-up interviews with specific users to delve deeper into their feedback and gain a richer understanding of their experiences. Change up your testing environment: If you're conducting remote testing, consider in-person sessions for a change. The physical environment can sometimes influence user behavior. Consider alternative testing methods: Explore alternative methods like A/B testing or eye-tracking studies to gather additional data and gain different perspectives on user behavior.

I would add an important phase: benchmarking. If we compare our current design with competitor products and services, we can be guided to discover areas of improvement and find new perspectives and inspiration.