User research is a crucial part of product management, as it helps you understand your users' needs, preferences, behaviors, and pain points. However, collecting user feedback is not enough. You also need to analyze the data and extract actionable insights that can inform your product decisions. How can you analyze user research data better? Here are some tips to help you.
Before you dive into the data, you need to have a clear idea of what you want to learn from it. What are the main questions you are trying to answer? What are the hypotheses you are testing? What are the success criteria for your research? Having a well-defined research goal will help you focus on the most relevant data and avoid getting lost in irrelevant details.
Over a period of time, especially at a big company, there are likely to be multiple user research efforts already completed. In addition, there may be internal SMEs you can interview who may not be in your product group or function and who have relevant domain knowledge from their previous lives. Combining all of it and synthesizing the different segments, needs contexts, and insights is often helpful. If you do this first, you can identify gaps that may exist in your knowledge, insights, trends you want to confirm, and areas you want to dig deeper. Don't be afraid to go back to first principles thinking but it is not necessary to reinvent the wheel each time. Your time spent on user research will be more productive if you do the pre-work.
Well, setting up a well-defined research goal is like having a GPS for navigating your data journey. It keeps you focused and prevents detours into irrelevant data. But remember, in the fluid world of product management, what’s “relevant” can change quickly. So, it’s crucial to set up “sentry points” in your data—indicators to reassess your original hypotheses and success criteria. Think of these as scouts on a battlefield, signaling when it’s time to pivot. Lastly, don’t forget to include cross-functional data for a 360-degree view. Product managements is not a solo act; incorporating insights from sales, marketing, and customer success can provide valuable context.
Depending on the type and amount of data you have, you may need to use different analysis methods. For example, if you have quantitative data, such as survey responses, ratings, or metrics, you may want to use statistical analysis, such as descriptive statistics, correlation, or regression. If you have qualitative data, such as interviews, observations, or open-ended questions, you may want to use thematic analysis, such as coding, categorizing, or clustering. You may also want to use mixed methods, such as triangulation, to combine and compare different types of data.
Be very clear about what you are trying to figure out and which research methods are appropriate. Quantitative methods used with sufficient sample size should focus on the most important quantitative hypotheses - e.g. segment size, competition, pricing etc. Qualitative methods such as contextual inquiry, user journey mapping, and value stream analysis inform different questions and hypotheses about the problem or solution. Be careful to use the right methods and not conflate them. E.g. 6 in-depth qualitative interviews show that tight integration with the ERP system is important for your solution and 4/6 use ERP system X. This doesn't support 66% market share of system X in your segment.
The type of data you have in hand practically dictates the analysis methods you should deploy, much like how you wouldn’t use a hammer for a job requiring a screwdriver. In the realm of product management, where you’re often juggling both quantitative and qualitative data, the right approach isn’t just a preference; it’s a necessity. But here’s the kicker: the real magic happens when you combine both. That’s what mixed methods like triangulation are for. For example, your statistical data might say a feature is not popular, but thematic analysis could reveal it’s because users find it hard to discover. This kind of multidimensional insight is gold in product management, allowing you to take calibrated, effective action.
Once, we encountered the challenge of managing a substantial dataset gathered from 8 or 10 interviewers as part of our end-user research. This dataset had over 100 data points. During the interviews, one person asked the questions, while the other person recorded notes directly on an online whiteboard. We used Mural. After completing all the interviews, we found it convenient to conduct a virtual brainstorming session and categorize the data points. What's even more advantageous is that many online whiteboard tools, such as Mural & Miro, provide pre-designed templates for various stages of the "Design Thinking" process, including 'Empathy Map' and 'User Journey.' These templates were very helpful in creating a highly detailed user persona.
One of the challenges of analyzing user research data is to make sense of the large and complex data sets. To make your analysis easier and more effective, you need to organize and visualize your data in a clear and meaningful way. You can use tools such as spreadsheets, charts, graphs, tables, or dashboards to sort, filter, group, or highlight your data. You can also use tools such as affinity diagrams, journey maps, personas, or empathy maps to synthesize and present your data in a user-centric way.
The next step is to look for patterns and insights in your data. Patterns are recurring or consistent themes, trends, or relationships that emerge from your data. Insights are the deeper or hidden meanings, implications, or opportunities that you can derive from your data. To find patterns and insights, you need to ask yourself questions such as: What are the common or unique user needs, preferences, behaviors, or pain points? What are the causes or effects of these user factors? What are the gaps or opportunities for improvement or innovation?
The final step is to validate and communicate your findings. Validation means checking the validity and reliability of your data and analysis. You can do this by cross-checking your data sources, methods, and results, as well as by seeking feedback from other researchers, stakeholders, or users. Communication means sharing your findings and recommendations with your team, clients, or users. You can do this by creating reports, presentations, or stories that highlight the key findings, insights, and action items.
While there are tools galore to consolidate and organize user research data m, it’s critical it is done in context of what success looks like following gathering this data. All stakeholders involved need to be aligned on how they will use the insights from this data and how it will further inform the next steps of the user research. Conducting this in a vacuum just for the sake of gathering data is a ineffective exercise.