Why A/B testing falls short, part 2: You don’t understand WHY!

This is part 2 of a 5-part series highlighting challenges you’re likely facing in your A/B testing strategy.

Check out part 1: You’re experimenting with your customers

Problem #2: You don’t understand WHY!

A/B testing comes in many shapes and sizes. Some tests can be as simple as testing whether people are more likely to click on a blue button versus a red one. However, we know things are usually more involved than that.

When launching an A/B test, we typically have multiple hypotheses and many variables. But at the end of the day, there is usually a single goal for the experiment to achieve - to convert visitors to customers, or at least to take them one step forward in that journey.

So you’ve launched the two pages, a month has passed, and ‘B’ performs better than ‘A’. 

What now? Do we introduce ‘C’ and do it all over again? Is ‘C’ likely to beat out ‘B’?

No clue!

We’re (sadly) relying on our gut feeling at this point and essentially guessing what new ideas we can introduce in order to improve.

Why, you ask? Because we don’t understand WHY!

We don’t understand why people are more likely to click on a blue button versus a red one.

We don’t understand why people feel their expectations are met.

We don’t understand why people feel the product price is fair or why the value proposition is attractive.

And probably most frustrating - we don’t understand why people are bouncing!

So what should we do?

Step one: Admit we have a problem!

We really don’t understand why.

Step two: Try to find a solution and accept that our current tools are just not cutting it.

Tomer Azenkot is the Chief Revenue Officer at WEVO.

Please follow @Tomer and @WEVO on LinkedIn to hear more ideas on how to improve digital experiences.

About WEVO:

WEVO is the first technology platform that optimizes digital experiences before you go live. Leveraging crowdsourced visitor insight and artificial intelligence, WEVO generates recommendations that have proven to significantly increase conversion. WEVO has successful experience working with Wells Fargo, Fidelity Investments, Blue Cross Blue Shield, Harvard University, Intuit and others.