Understanding Prioritization Frameworks

Ideas are everywhere around us, they can be generated from our different experiences ranging from insights gained from experiments to conversations made with people and beyond. What happens when you and your team come up with 100 ideas on what to test to optimize your website, how long will it take you to test 100 different ideas? What is the certainty that the tested ideas will have an impact and increase conversion rate? What are you and your team going to test first or next?

Situations like this justifies why you need a prioritization framework. Prioritizing your test the right way makes your optimization impactful and powerful.

Some of the reasons why you need a prioritization process includes:

-        Prioritization helps to manage stakeholders

-        Prioritization brings transparency to optimization

-        Prioritization builds trust from the company

-        Prioritization empowers employees to share ideas that they feel will have an impact

In simple terms, prioritization is a critical part of building an efficient, impactful testing program.

Whenever you hear somebody asking your team, “What should we test next?” This is often a clear sign that your optimization process is completely broken. If you have no idea what to test, it’s a symptom of a missing process because optimization is a process.

Creating a framework to make smarter choices and thinking deeply about the key factors that can help your business grow and will make your optimization process successful in the long term.

Before you set up your prioritization/ build a framework, there are key things you need to know:

1.  Know your destination/Know what you want:

This involves:

Testing profile: Are you a beginner or advance in optimization? Are you going to test incremental changes or bold new features?

Product strategy: Where do you want to take your product? When you know where you want to take your product, you know how to optimize your test.

Optimization goals: This could be the conversion rate or any other metric you want to focus on.

2. Know who you are:

Get your analytics right: Where are the pages you need to work the most? Where in the funnel are you having high drop-offs and high performance?

Talk to your customers: You need to know what they like about your product and what they don’t like.

Look at your competitors: Look at what they offer their customers and use that to find where your gaps are.

3. Structure your ideas:

Translate ideas into hypothesis: Now that you have ideas on what to test, it’s time to turn them into hypothesis. So that you know what you want to optimize.

Centralize: You can have a central repository where you collect all this information and even color code to allocate the level of importance. More like rank and sort your ideas. You can use Google Sheets or Msword Excel.

Organize and tag: Organize based on all the teams that will be involved. Tag every member and ensure they follow through with their roles in the optimization process.

Defining prioritization rules:

How do you decide what to test first and what you will test after that? What are you trying to achieve when building your prioritization framework?

First, it all comes down to value and effort. You need to identify the drivers of value and effort. This means, how valuable the test is and how much effort is needed to build it. Once you get effort and value right, the rest is easy. If you get both right, you’re able to make the right decision with your different ideas.

Prioritization Frameworks

All the frameworks break down value and/or effort into sub-elements. Here are the most common:

1. Value

Reach: It is- how many people will be impacted by your test. For example, if it’s only for returning customers or customers from a specific channel, it might not make much of a difference.

Lift: It is - reasons to expect a strong reaction. For people who will be part of the test, do you expect to see a dramatic change in behavior?

Strategic fit: Is the idea aligned with strategy? Your optimization program should be the way you want to get to where you want to be. So, an idea that fits into that should be a priority for you.

The three underlined pillars of value (reach, lift and strategic fit).

2. Effort

Design: How many designers and how long will it take to put the design in place (if any).

Engineering: Which developer is going to commit to the project? How much time will be required for implementation?

Coordination: Who is handling this project and how many people will they manage to execute it?

Types of prioritization frameworks:

There are different types of prioritization frameworks. All of these frameworks have interesting ways to break down value and effort. One of the challenges is that it is still arbitrary to put number on them.

Lift is very much your own intuition and it’s hard to make it objective. Since it is a big impact in how you are going to rule your test, try to find something that takes some of the emotion and the personal bias out of the way to make it a bit more structured. For the other pieces, people might come with very different result if they are asked to score.

PIE Framework (Here, value is broken down into potential and importance, while effort is bundled together as ease)

Brian Eisenberg rules (Breaking down effort into resources and time)

Monetate model (Breaking down effort into creative and development)

I will expatiate on two frameworks from the Growth Marketing mini-degree at CXL Institute. The PXL framework by CXL and the Points Model by Hotwire.

Hotwire Framework: The Points Model

Hotwire created a systematic, scalable framework that eliminates emotion and personal bias. They broke down each sub-element into specific rules, to remove personal bias. This framework is used within the organisation.

Value: Define binary rules that are: objective (yes/no), discriminates ideas -something that not all your ideas will have in common, leverage your knowledge through research, analytics, competitors.

This is broken down into three:

Reach: How many people will be impacted?

Lift: Do you expect to see a dramatic change in the behavior of users?

Strategic fit: Is the idea aligned with strategy? Your optimization program should be the way you want to get to where you want to be. So, an idea that fits into that should be a priority for you.

Effort: Define effort rules that: represent your bottlenecks - do your research to see what it will take your test to run and ensure that the requirement is aligned with your company’s existing sizing.

Creative: How much work will you need from your design team?

Development: How much effort from engineering and what is the timeline?

Coordination: This is basically supervision and bringing everyone together to get the test to run.

Points Model Breakdown

For each rule, 1 point is assigned if it will have a significant influence on the product and 0 points if the influence is almost non-existent.

Read more about the hotwire framework here

PXL Framework

This framework has three benefits:

1.   It makes any “potential” or “impact” rating more objective.

2.   It helps to foster a data-informed culture.

3.   It makes the “ease of implementation” rating more objective.

A good test idea is one that can impact user behavior. So instead of guessing what the impact might be, this framework asks you a set of questions about it:

Is the change above the fold?

This means the changes are noticed by more people which leads to a likelihood of the test having an impact.

Is the change noticeable in under 5 seconds?

If it’s not, then it’s less likely to have an impact.

Does it add or remove anything?

Will significant changes like removing bottlenecks such as reducing the requirements on a form that is causing huge drop-off or adding key information to the landing page.

Does the test run on high traffic pages?

The pages on your website with high performance, will the tests run there? Running tests on high traffic pages means higher conversion rates and in the long run, higher revenue.

The PXL model asks every frontline team member to bring data:

·      Is it addressing an issue discovered via user testing?

·      Is it addressing an issue discovered via qualitative feedback (polls, surveys, interviews)?

·      Is the hypothesis supported by mouse tracking heat maps like Hotjar or eye-tracking?

·      Is it addressing insights found via digital analytics?

This week’s learning at CXL Institute required in-depth concentration. The learning on prioritization will help me even beyond work.