User research is a vital part of product management, as it helps you understand your users' needs, preferences, and behaviors. But how do you choose the right method for your project? In this article, we'll compare qualitative and quantitative user research, and discuss their pros and cons for different scenarios.
Qualitative research is a method that explores the why and how of user behavior, using techniques such as interviews, observations, or focus groups. It allows you to gain rich insights into user motivations, emotions, and pain points, as well as generate new ideas and hypotheses. However, qualitative research also has some limitations, such as being time-consuming, subjective, and hard to generalize.
Often, your first form of insights will actually be data on behavior of customer ms and users. For example, “New members don’t retain if they don’t complete their profile” is a key insight, but it leads to the question of why they don’t complete their profile. Qualitative research can be a great place to build hypotheses to explain user and customer behavior. The super power of Qual is understanding the thoughts and emotions behind that behavior. These can be validated in quant, so you can calculate growth opportunity sizes. From Qual, make sure to get the user’s needs/intent. You might find that they’re using your product for an entirely different reason that you anticipated, opening up new markets for your business.
Bridging the "say vs. do" gap is essential to really understand human behaviours. People are very good at answering questions in interviews one way, and then doing something completely different in real life. The best way to really understand how people behave or use products, is to observe them in their natural habitat - whether that's through ethnographic research, experiments or testing with prototypes. Whatever methods you use, observing users in real-life scenarios is a critical ingredient to meaningful insights.
Quantitative research is a method that measures the what and how much of user behavior, using techniques such as surveys, analytics, or experiments. It allows you to collect large amounts of data, test hypotheses, and quantify user satisfaction, engagement, or conversion. However, quantitative research also has some drawbacks, such as being costly, rigid, and unable to explain the underlying reasons for user behavior.
Qualitative research is an invaluable tool for the early stages of product development, as it can help you explore user problems, needs, and expectations, and generate ideas for solutions. It can also be used to validate assumptions, uncover user feedback, and refine product features and design. Qualitative research can help you answer questions such as what are the main challenges and goals of your users, how do they currently solve their problems or fulfill their needs, what are their preferences, attitudes, and emotions towards your product or similar products, and how do they use your product or prototype, and what are their pain points and delights?
Another advantage of qualitative research is generating ideas for solutions. Techniques like user interviews, focus groups, and surveys, generate qualitative data product managers can use to gain direct access to users' thoughts, ideas, and feedback. These insights help validate assumptions, uncover new user needs, and refine product features and design. By engaging with users on a personal level, qualitative research enables a collaborative and iterative approach to product development, fostering empathy and connection with the user base.
Quantitative research is ideal for the later stages of product development, when you need to measure user behavior, performance, and outcomes, and optimize your product for scale and growth. It can also help you evaluate your product-market fit, compare different alternatives, and identify trends and patterns. Through quantitative research, you can answer questions such as how many users are using your product, how satisfied they are, what the key metrics and indicators of success are, and how different variables or factors affect user behavior or outcomes.
Qualitative and quantitative research are not mutually exclusive, but rather complementary methods that can provide a holistic view of your users and your product. By combining them, you can leverage the strengths of both methods and overcome their limitations. For instance, you can use qualitative research to generate hypotheses and then use quantitative research to test them. Additionally, you can use quantitative research to identify user segments and then use qualitative research to dive deeper into their needs and behaviors. To combine qualitative and quantitative research, you should start with qualitative research to explore user needs and problems, and then use quantitative research to measure and validate them. Additionally, you can use mixed methods, such as surveys with open-ended questions, or analytics with user feedback, to collect both qualitative and quantitative data. Finally, you should triangulate your data from different sources and methods, and look for consistencies and discrepancies.
When budgets and resourcing allow, always try to combine Qual and Quant sources. The Qual can set the table with “what do our users and customers want? How are they reacting to the current offering” and the Quant data can inform scope of the problem and inform opportunity sizings. Research reports often need to include a “why this matters” section, to ensure they are seen as actionable. Quant excels at this step, but can sometimes be less memorable. “Our score was 34 and our competitors were at 42” doesn’t have the emotional resonance of quotes and clips from Qual, among your internal audience. Finally, make sure to include the behavioral data itself. Avoid having “our goal is to improve this score” as your takeaway.
You may experience different barriers to qualitative and quantitative research if you're B2B2C vs B2C. There may be restrictions on the types of data you can collect based on industry or a customers stance on privacy and you'll need to navigate it. Consider using panels instead for qual and quant, diaries, etc to ensure you've captured needs and jobs to be done.
Another factor to consider is competitive research. For the majority of companies that are fast followers (or not so fast followers) rather than leading edge, comparisons of product / solution features can give you a head start of what to new features to focus the qualitative and quantitative research on. Keep in mind that the ultimate behavior of these new features is typically constrained by factors such as cost and the regulatory environment.