How do you validate product hypotheses? This is a crucial question for product innovators who want to build solutions that customers actually need and want. In this article, you will learn some practical techniques to test your assumptions and get feedback from your target market before investing too much time and money.
Product hypotheses are statements that express your beliefs about the problem you are solving, the solution you are offering, and the value you are creating for your customers. They are based on your research, insights, and intuition, but they are not facts. They need to be validated or invalidated through experiments and evidence.
Product hypotheses serve as your initial roadmap in the uncertain terrain of a new venture. They are testable statements that encapsulate your beliefs about the problem you aim to solve and the solution you intend to offer.
There are 5 variables you can hypothesize on, and it should be validated in this order 1. Market 2. Problem 3. Solution / value proposition / use case 4. Product / service 5. Feature sets
To formulate product hypotheses, you need to be clear about your product vision, goals, and strategy. You also need to understand your customer segments, their needs, pains, and gains, and how your solution fits into their context. A good way to structure your hypotheses is to use the following format:
We believe that [customer segment] has [problem/need] and will [outcome/benefit] if we [solution/feature].
For example:
We believe that busy professionals have difficulty finding healthy and convenient meals and will save time and money if we deliver personalized meal kits to their door.
When framing product hypothesis don't mention the feature, but mention the capability enabled by the feature sets instead. This is why using jobs-to-be-done works better, using this frame: 1. When {Current negative state + problems} 2. I want to {capability} 3. So that {positive outcomes} You want to find the capability that will help your user/customer to go from current state to the future state without mentioning the product or feature.
You will probably have multiple hypotheses to test, but you cannot test them all at once. You need to prioritize them based on two criteria: uncertainty and impact. Uncertainty means how confident you are that your hypothesis is true or false. Impact means how much value your hypothesis will create or destroy for your customers and your business. You can use a matrix to rank your hypotheses from high to low uncertainty and impact, and focus on the ones that are in the top right quadrant, as they are the most risky and rewarding.
The guiding question for how critical a hypothesis is: “What has to be true?” What has to be true for my business model to work at all? Combine that with the level of uncertainty and you get the most critical hypothesis. In many cases it will be the underlying customer problem, but it can also be key partners, technologies or go-to-market strategies.
I’m not quite sure that the solution above mentioned works for all cases. For instance, if what you want to study (response variable) is the result of a combination of multiple variables, then, isolating and test the impact these variables one by one may not help you assess the interaction among them. In that case, a design of experiments that could help assess the impact of all the independent variables plus their combination may be a better option. If properly conducted, the resulting equation will help assess the magnitude of the impact of each variable and the magnitude of the impact of the combination of variables
Prioritizing product hypotheses is a balancing act between risk and reward. While it's tempting to go after low-hanging fruit, sometimes tackling the riskier hypotheses can yield more significant insights and competitive advantages. Consider not just the potential impact on your business, but also the speed at which you can test and the resources required. Sometimes quick wins can fund more extensive, longer-term experiments.
If you haven't spoken or observed customers through experiments about the product hypotheses that you put together, you got to make some bets which problems are worth solving. Choose the product hypothesis based on problems that are - Ubiquitous: it’s everywhere - Growing: it gets bigger as time goes by - Urgent: it’s catastrophic if it doesn’t get solved - Expensive: it’s expensive to get the job done - Mandatory: required by law to comply - Frequent: occurs frequently - Fragmented: broken in many parts Once you selected them then you got to validate them with your market.
Once you have prioritized your hypotheses, you need to design experiments to test them. Experiments are ways to expose your hypotheses to real or potential customers and measure their responses. Depending on your stage of product development, you can use interviews, surveys, landing pages, prototypes, MVPs, or pilots. For a successful experiment, it is essential to have a clear and measurable success metric that indicates whether your hypothesis is validated or invalidated. Additionally, ensure that the experiment requires minimal resources and time to run. Also verify that the sample of customers matches your target segment and that the data collection and analysis is reliable and ethical.
PRETOTYPE AND TEST LIKE TESLA As innovation facilitator of 100+ projects, I like to share how Tesla interested buyers for their first car. In 2003, the founders of Tesla had an ambitious idea (an all-electric 2-door sports car). In order to convince potential buyers to order its car, Tesla created a pretotype of what the car would look like. Lotus supplied Tesla with a ‘glider’ Elise - which was filled with models of key components. This was not a prototype, because the vehicle didn’t function. Tesla was able to show buyers a close proxy for the final design. Tesla asked “Will you put down a $5,000 deposit to have it built to order? From this test Tesla secured several hundred deposits, a non-trivial result to reassure their Investors
Designing product experiments is where the rubber meets the road. It's essential to isolate variables to understand cause and effect clearly. However, life is not a controlled lab experiment; external factors will influence outcomes. Therefore, use a mix of qualitative and quantitative methods to gather a more holistic view of how your product is performing. Also, don't underestimate the power of A/B testing to provide quick, actionable insights.
Devising the right experiments is based on 1. What assumptions need to be experimented? - Market - Problem - Solution - Product 2. What experiment formats to use? - Pre-sales for market, problem, solution, product - Workshop for market, problem, solution, product - Interview for market, problem, solution - User testing for product - Etc
The final step of validating product hypotheses is to learn from your experiments and use the results to inform your next actions. You should compare your actual outcomes with your expected outcomes and see if they match or not. If they match, you can consider your hypothesis validated and move on to the next one or iterate on your solution. If they do not match, you can consider your hypothesis invalidated and either pivot to a new direction or refine your hypothesis and test again. You should document your learnings and share them with your team and stakeholders.
Validating product hypotheses is an essential skill for product innovators who want to create solutions that customers love. By following these techniques, you can reduce uncertainty, increase impact, and improve your product decisions.
Learning from product experiments is not just about whether you were right or wrong; it's about understanding why. Dive deep into the data to look for patterns or anomalies that can offer additional insights. Was the hypothesis flawed, or was the execution lacking? Sometimes a 'failed' experiment can offer more valuable insights than a successful one. The key is to keep an open mind and be prepared to go where the data leads you.
You're essentially testing these 5 variables and see whether they are ALL aligned together. If you do this right you are creating a good start for PMF. 1. Market 2. Problem 3. Solution / value proposition / use case 4. Product / service 5. Feature sets
ABV: Always be validating. So build close relationships with your users and customers. As you release more features or create new initiatives, you always got to find leading indicators as evidences towards product-market fit. Product innovation is never done.