Growth experiments are essential for finding new ways to scale your business and reach your target audience. But how do you decide which experiments to run, when to run them, and how to measure their results? In this article, you will learn a simple framework to prioritize growth experiments for maximum impact and minimal risk.
The ICE score is a popular method for prioritizing growth experiments. This numerical rating is based on three criteria: Impact, Confidence, and Ease. To determine the Impact, consider how much the experiment will affect your key metrics, such as revenue, retention, or conversion. For Confidence, assess how sure you are that the experiment will have the expected impact based on data, research, or intuition. Lastly, consider the Ease, which is how easy or difficult it is to implement the experiment in terms of time, resources, and complexity. To calculate the ICE score, multiply the ratings for each criterion (from 1 to 10) and divide by 3. The higher the score, the higher the priority of the experiment.
The ICE score was pivotal in our decision to launch Your Digital Future. It scored highly in our quarterly director's review, showcasing it as a standout for its potential impact, our confidence in its success, and the ease of integration with our existing services. This framework guides us to make informed, strategic decisions.
Another common method to prioritize growth experiments is the PIE score, which stands for Potential, Importance, and Ease. This score is similar to the ICE score, but it focuses on two different criteria: Potential, which considers the room for improvement compared to the current performance of the area being tested; and Importance, which evaluates how valuable the area is to the overall business goals and strategy. The PIE score is calculated by multiplying the ratings for each criterion (from 1 to 10) and dividing by 3. Thus, the higher the PIE score, the higher priority of the experiment.
The ICE framework offers a simple model for prioritizing experiments. However, its simplicity is also a weakness - the criteria lack nuance. For example, long-term brand impact may outweigh short-term revenue gains. Overweighting confidence risks stagnation. Ease should consider technical debt and downstream effects. While the intent is good, ICE's reductive scoring obscures important complexities. Leaders need to supplement ICE with deeper analysis when evaluating experiments. More perspective on ICE's limitations and balancing it with other inputs would strengthen this advice. Overall a helpful starting point but insufficient on its own.
A third method to prioritize growth experiments is the RICE score, which stands for Reach, Impact, Confidence, and Effort. The RICE score is a variation of the ICE score, but it adds one more criterion:
- Reach: How many people will the experiment affect, either directly or indirectly, over a certain period of time?
The RICE score is calculated by multiplying the ratings for Reach, Impact, and Confidence (from 1 to 10) and dividing by the rating for Effort (also from 1 to 10). The higher the RICE score, the higher the priority of the experiment.
Each of these methods has its own advantages and disadvantages, depending on your context and goals. For instance, the ICE score is simple and intuitive, but it can be subjective and biased. The PIE score is useful for aligning experiments with business strategy, yet it can be vague and hard to measure. On the other hand, the RICE score is comprehensive and data-driven, yet it can be complex and time-consuming. Moreover, none of these scores take into account the size of the audience, duration of the experiment, certainty or effort of the experiment, value or relevance of the area being tested.
Balancing the pros and cons of methods like ICE is key. It’s straightforward yet subjective. To mitigate bias at Optix, we blend quantitative data with team insights. This ensures a balanced approach to prioritising growth experiments. Tailor these frameworks to your business needs for best results. Good luck!
When it comes to prioritizing growth experiments, there is no one-size-fits-all approach. The best method for your situation depends on the data availability and quality you have to support your assumptions and hypotheses, the resources and constraints you have to execute and analyze your experiments, and the goals and objectives you want to achieve. To choose the best method, you need to consider these factors and weigh them against the pros and cons of each method. Additionally, it is important to test and validate your assumptions and hypotheses with real data and feedback in order to adjust your priorities accordingly. Ultimately, prioritizing growth experiments is an ongoing process that requires experimentation and learning.
Prioritising growth experiments is more art than science, contingent on your data's robustness, resource availability, and specific goals. The right approach is a balance of these factors against the strengths and weaknesses of methods like ICE. Crucially, validate assumptions with real-world testing and feedback. It's a dynamic, iterative process, with learning as valuable as outcomes. At Optix, this is our practice — constant testing, learning, and adapting for optimal results.
At the end of the day, it's great to have tools for evaluation, prioritization etc. However these should never be a blocker for action. I usually try to internalize the principles behind these tools but at the end of the day a simple excel sheet and notion to do list are my best friends, better to be biased to action than perfection. But also remember to fail fast and move on to the next thing...
When using any method like this to prioritise growth projects its important to remember to ask, are you measuring what you say you are? Sometimes the chosen measures don't actually reflect what they are supposed to assess. Also 'who' decides the measure and weights is very important too. Making sure these processes are free of any negative politics and conflict is important if a leader is to be successful in using these methods.