Testing 1..2..3 : Review

Welcome to another insightful article ??

If this is the first article, you are reading authored by me, then you need to check out my last three articles here. This way you are a step closer to being the best marketing professional. Everything I am sharing with you is from my growth marketing minidegree course offered by CXL Institute.

I need you to think deeply about the question I am about to ask.

Do you test your marketing strategies before deploying them or do you just push out the strategies with hopes of getting positive results? If you test your strategies by doing a test run or having a demo done before making it public, you are doing something right! If you do not test it, why?

In my last article, I wrote briefly about research and testing. Now, I would be expanding on A/B testing and sharing what I have learnt about this form of testing. If you have heard about A/B testing and you are still wondering what it is, you have come to the right place.

Let's go back a bit to 2010. What a year! 2010 was an amazing year in the marketing industry and I would tell you why. This was when A/B testing really started but many people did not know how to apply it and it was just a series of experiments after experiments with little or no results. They did not know how to apply it but by 2013, people then found a need to run experiments on applications, websites, and the likes.

Now, here we are watching companies run tests before implementing their strategies. Companies have tools which they use to run their tests and they are now embedding optimization frameworks in their servers. Platforms like google optimizer allows people to run experiments with drag and drop interfaces frameworks, personalization’s, and servers.

A/B testing allows you run effective strategies. You need to know for sure that you are going to be doing something that has an impact and not just based on a few opinions. This is how you trust your final decisions. You should run your A/B tests weekly and the tests should be run weekly because customers have a buying cycle, and it takes at least 7 – 8 days for an average customer to buy a product which means you cannot experiment for a few days. A/B tests are used for deployment and research. When you deploy something on your website it can be for various reasons like a new feature, a new update, etc. You need to deploy it as an experiment and then you learn the impact of what you are deploying. Looking at research, you want to know which elements are making impacts on your website and which elements are not.

Even Jeff Bezos acknowledges the use of experiments. He says the success at Amazon is a function of how many experiments they do per year, per month, per week and per day.

The secret to A/B testing is in ROAR. Risk, Optimization, Automation, and Re-think. Its simple! I see it as taking a risk to try out something that you get to optimize and then integrate into your system. If it does not work out, then you re-think it. Risk and Automation are your most important stages.

Now, check your website analytics, what is your conversion rate like? If you have less than 1,000 conversions per month then you should not be running any test. To run tests on your website, you need to have more than 1,000 conversions in a month. This is when you get the best results. Conversions are transactions, lands, clicks, any goal you are trying to optimize. It does not have to be final conversions.

Just as every business has KPIs they are trying to meet, A/B testing also has KPIs and these are referred to as goal metrics. These are the goal metrics you should focus on in order of priority; potential lifetime value, revenue per use, transactions, behavior and click being the least metric.

Immature companies use click a lot and sometimes behavior but if you really know what you are doing then you should focus on transactions. More mature companies can shift to revenue per user or go as far as potential lifetime value. Ensure you have a goal metric that is the same for everyone across all departments in your company and this is called your overall evaluation criterion.

As I begin to come to the end of this article, I would be sharing some important points with you. I discovered there is a 6V model for conducting research that helps you get insights for your A/B tests. The Vs are the value of your company, versus (your competition), the view of your customer, voice of the customer, verified data and validated data.

You need to make observations of all these so that you have something to base your hypothesis on then you experiment he hypothesis to know what is true or not and this is what then shapes your theories.

So, you create – test – analyze.

After doing the research based on your 6Vs, you write a hypothesis before you make the growth experiment. Note that the 6Vs research is not technical but based on desk research which is findings you can have easy access to. When you write down your hypothesis, it helps you brief everyone that would be working with you on the direction of the solution.

The format here is easy/ Research- hypothesis writing – experiment idea – problem – proposed solution – predicted outcome.

Yes! I learnt how to prioritize my A/B test and I just must share this. I discovered three models: the PIE model, ICE model and PIPE model.

From the pipe model, Potential is the chance of the hypothesis being true, Impact is looking at when the hypothesis would have a bigger effect, Power is the chance of finding a significant outcome and Easy looks at how easy it is to implement. Ensure you pick a prioritization model that suits your company.

If you have not taken anything away from this article, of which I am sure you have, note this. Design just one challenge, run two simultaneous experiments on the full population of the website and then you develop the A/B test you would work with. Once you start getting significant results then you can stop your A/B test.

Optimization is about effectiveness. Organizations are mostly focusing on efficiency but as a growth marketer, focus on efficiency and effectiveness and this helps you maximize the number of positive impacts you get with the same resources.

Here are few concepts every digital marketer should know, and they help you evaluate test results for A/B testing.

·        Population: all potential users in a group or things we want to measure

·        Sample statistics: used to make inferences about population parameters

·        The shape of data influences the test duration and sample size needed

·        Confidence interval: the amount of error allowed in a test