How to use the 6V Conversion Canvas to get insights for A/B tests. - Review

A/B (or split) testing is the method of showing 2 variants of the same web page to 2 different audiences, to discover which of them converts better.

In the world of websites and apps, the number of visitors one is able to attract, is proportional to the opportunities to convert them into paying customers. Of course, nurturing the relationship with existing ones, will obviously also grow the business bottom line.

And it is one's conversion funnel that decides whether one's website gets good traffic and if it converts more visitors.

Every business owner's goal is to make visitors take the action that they want (conversion), and the higher the rate at which visitors take the intended action, the higher the conversion (rate of conversion). Furthermore, the more optimized the journey of the visitor, the higher the likelihood of conversion.

Now, one of the best ways to optimize one's user experience is to execute A/B testing!

As standard practice, the variant with the most conversions will be the winning one, and I have found out a lot efficient more ways to get started with A/B testing during my mini-degree study at CXL Institute.

One of the interesting things I learned was the history of A/B testing. Truth is, A/B testing has been a human phenomenon, since even the days of Genesis, in the Bible: the story of Cain and Abel comes to mind.

Fast forwarding to the advent of the Internet, from 1995 to 2020, the concept of A/B testing has evolved to increasing levels of effectiveness. Because the efficiency levels of A/B testing are co-related to the rate of conversion, it should be considered as an essential component of growth marketing planning.

The value of A/B testing

A/B testing has really proven to be the silver bullet for many businesses, as the process allows for quick, small tests (EFFICIENCY) that deliver those small victories (EFFECTIVENESS) that become drivers to building scalable systems. The value of A/B testing can be viewed in two (2) ways:

• To understand what EFFECTIVENESS can derived out of EFFICIENCY
• To understand the position of A/B testing within the hierarchy of evidence

In these processes, the underlying principles are based on the Random Control Testing (popularized by the health sector), which allows for effectiveness to be placed on top of efficiency.

When to use A/B testing

To understand in what situations A/B testing can be valuable, and how it should be applied in those situations, one has to consider this stages of conversion optimization:

• Research - to find signals that will inform what is worth hypothesizing and what is not
• Learn - to optimize small wins into systems that loop to scale
• Deploy - the real wins and build into sustainable processes

The first thing to consider when planning to A/B test, is conducting a thorough research on one's website performance. Data will have to be collected on all activities related to user visits, traffic sources, best performing pages and what the various conversion goals of the pages are.

The other thing to consider is whether one has enough data to conduct A/B tests. To achieve that, let's look at the optimization phases, using the ROAR (Risk, Optimization, Automation, Rethink) model.

One needs to calculate if they have enough statistical power to conduct A/B testing. It projects the likelihood that an experiment will detect an effect when there is an effect to be detected.

Between the risk and optimization stage, one needs a monthly minimum of 1,000 conversions to have enough data to hypothesize for optimization. To automate, a conversation of 10,000 per month is required.

This is dependent on sample size, effect size and significance level.

It is also imperative that to achieve significance, one needs to test against a high enough significance level (90% or 95%), otherwise one will declare a winner when in reality there isn't an effect (false positives).

KPIs to prioritize

To derive the best out of one's A/B testing endeavours, one has to scale the goal metrics to measure, keeping in mind what to prioritize:

From the above, clicks should be the least valued metric, while potential customer lifetime value should be the holy grail metric to be measured/projected.

Leveraging 6V Conversion Canvas to master A/B testing

This is where the importance of the 6V Conversion Canvas becomes apparent, and that is the crux of this article.

So now, let's look at the various segments of the conversion canvas in detail, and explore the various processes required:

VALUE - See KPIs to prioritize above.

VERSUS - This helps one to discover who the competition is. Searching with the relevant keywords will reveal who are ranking for those category keywords, and enables one to gauge what they are up against and to start strategizing how to win.

Alexa also helps to discover audience overlap with other websites, which is super imperative if one is to have a broad view of the totality of their potential audience size.

Another way to research competition is to visit competitor's website, and buy from them; literally go through the customer journey.

Wachete is another tool that can be utilized to research the competition. It can be used to track website changes on both one's and competitor's website.

VIEW - This utilizes web analytics to get to know user behaviour, and answer questions like:

• Where do visitors start on the site?
• Where they come from
• What the flow of those visits are
• Are there any notable differences between segments and products?
• The behaviour on the most important pages
• Difference between existing and new customers
• Difference per device

The view is also used to create behavioural segments. So, for instance, a typical e-commerce flow will be something like this:

• All users on your website with enough time to take an action
• All users on your website with at least some interaction
• All users who have heavy interaction with your website
• All users with clear intent to buy
• All users willing to buy
• All users who buy
• All users who return to buy

Another way to view the activities or performance of one's website, is to utilize heat and scroll map tools. With all these, one can report on:

• Users per segment
• Conversion and time it takes to move from segment to segment
• What are the important pages where decisions are made
• What is the detailed behaviour (for segments) on these pages

VOICE - This is to help one "hear" the sentiments of customers and prospects. To get a clear sense of the voice of the market, talk to customers service, check out live chat logs and look at social media feedback, through your community moderators and managers. Also, utilize the power of website surveys and feedback forms. One can also engage in user research through customer interviews, focus groups and usability studies.

The best thing to do is to use this exercise to get the why, but never to ask the users to tell one why. This is because, they will probably come up with something that is made up and not helpful to discovering they whys. Instead one can recruit testers and pretest one's assumptions, by determining what type of test one wants to run.

VERIFIED - Scientific literature from subject-matter experts who have published writings are also another key way to research for A/B testing. Semantic Scholar, Google Scholar and Deepdyve are some of the tools available to verify one's findings. The importance of these tools is that, it helps one to answer questions like:

• What do we know from scientific literature?
• What's the general best practice about decision-making processes
• And specifically about the type of products sold

VALIDATED - At this stage, one focuses on going through experiments that have been conducted already to get a good insight into customer behaviour. It helps to find out if there are any validated hypothesis, or if there are any empty element experiments.

The goal of the customer behaviour study is to gain insights into the most important customer journeys, understand the basic user behaviour and input for setting hypothesis.

It's also imperative that one makes assumptions on the ability and motivation of users to take the required actions.

According to BJ Fogg, motivation, ability and a prompt must converge for an action to happen.

When any one of these three (3) is missing, then the trigger that aims to engender action fails. Reference https://www.growthengineering.co.uk/bj-foggs-behavior-model/ for further reading.

Finally, one also needs to make assumptions on emotion and ratio. Daniel Kahneman's Thinking Fast & Slow explains two (2) systems of thought that influence the way people think and make decisions.

In effect, the results of thinking fast and slow have their respective repercussions, which makes for interesting hypothesizing, when researching consumer behaviour.

All these go to confirm that belongingness and conformity can be used to hypothesize how users are moved from strangers to advocates.

In concluding, A/B testing can be a very rewarding exercise, if one makes the effort to research thoroughly, leveraging the various processes within the 6V Conversion Canvas. I have learned a lot during this impactful time at CXL Institute, and look forward to sharing more nuggets with you, over the next 4 (four) weeks.