What's your process for testing innovation ideas?

Starting with the problem is key to focusing your solutions on core human needs. But when running business experiments, you're most likely going to be testing solutions, not problems. Here's how to do that without losing sight of your problem. Encourage the team to generate lots of solutions to your defined problem. Go broad and don't filter. After you have a big list, select 2-3 ideas that are feasible, viable, desirable, and usable. These are the ideas that you want to "multi-track" throughout the experimentation process. That way, you won't be tied to a single solution, thus reducing confirmation bias. Keep an eye on your problem, and feel free to evolve or even combine your 2-3 ideas into a final solution after testing.

Starting your experiment design with a hypothesis can lead to overly general experiments that test non-crucial aspects of the idea. A better way to experiment is to de-risk a solution idea by pulling out the assumptions. An assumption is accepted as true or sure to happen without proof. To find an assumption, ask: "What must be true for this idea to succeed?" Try to surface assumptions about the idea's feasibility, viability, desirability, and usability. Prioritize the risky assumptions most likely to make an idea fail. Do this by choosing ones with little evidence and a high potential to kill the idea (use an Assumption Map here). Once you have the riskiest assumptions, you can form your hypotheses and experiment.

Defining the metrics and success criteria and implementing ways of collecting the data (e.g. Google Analytics on your landing page) are absolute pre-requisites before launching an experiment. Too many corporate innovation teams (and startups) feel compelled to skip this step - usually due to an urge to "move fast". But if you wait until after the experiment, the temptation is to define metrics and success criteria in a way that supports whatever you want them to. When there's pressure to show positive results (rather than the market's honest feedback) teams tend to produce experiment results that look great at first. But this is extremely harmful and costly long-term, because you're probably building something the market doesn't want.

When we analyze experiment data alone, we limit the insights that we can form. Try to bring the whole team into the experimentation process so that you can reduce the bias of your analysis. Try running an "experiment retro" and gather the experiment data so the team can form insights and learnings. I like to frame the analysis discussion around three things: observations, inferences, and recommendations. Observations ensure we're noticing all the outliers and surprises during the experiment. Inferences force us to formalize what meaning we form while looking at this data. Recommendations allow us to make suggestions for future experiments to improve our experimentation process in a virtuous cycle.