Why A/B testing falls short, part 4: You’re optimizing for the wrong persona
This is part 4 of a 5-part series highlighting challenges you’re likely facing in your A/B testing strategy. Check out the first 3 parts of this series:
Part 1 You’re experimenting with your customers
Part 2 You don’t understand WHY!
Part 3 You need to wait a looong time
Problem #4: You’re optimizing for the wrong persona
Much of the optimizing we’re doing when running A/B tests is focused on the wrong audience.
Let’s think about this for a moment: An A/B test, or any live traffic analysis for that matter, is informing us on what our website visitors have done. Did they choose option ‘A’ or option ‘B’? Which of the options yielded the higher conversion rate?
We don’t know much about them. We don’t even seem to care, when it comes to our A/B test results. We simply want to know which option, of the ones put in front of our website visitors, drove better results. So why is it that we spend so much time and effort on defining personas and trying to understand our highest lifetime value customers in all the other marketing activities?
Personalization may be the Holy Grail, but...
It’s not always possible. And even if it is, are we really doing it effectively?
Assuming for a minute that we have the capability to personalize, we typically find ourselves having limited data to segment our actual audience anyway.
For new customer acquisition, website visitors show up unannounced and anonymous. We’re lucky to know the traffic source (i.e. what they did before browsing to our site), but is that enough to understand their expectations and needs? Probably not. So who should we optimize for?
Not all conversions are created equal (for most businesses)
We typically have a known persona who is the most valuable buyer for our product or service. More often than not, this is our highest lifetime value customer and precisely who we need to optimize for.
A/B testing on the other hand, is relying on whoever shows up. We’re not controlling the optimization by targeting who we’d like to convert, instead, we aim for the highest conversion rate, period. Furthermore, to add insult to injury, most conversions don’t come free. We’re paying with our ad campaigns, we’re paying when the sales teams are engaging, and we’re paying with our customer support and cost of goods sold.
Now more than ever, reducing acquisition costs as we grow our businesses is key. Google and Facebook are lining their pockets with increasing cost-per-clicks (CPC). Customer acquisition costs (CAC) continue to rise for most. Improving margins by getting the right customers in the door, and then out with a full cart should be everyone’s top priority.
Tomer Azenkot is the Chief Revenue Officer at WEVO.
Please follow @Tomer and @WEVO on LinkedIn to hear more ideas on how to improve digital experiences.
About WEVO:
WEVO is the first technology platform that optimizes digital experiences before you go live. Leveraging crowdsourced visitor insight and artificial intelligence, WEVO generates recommendations that have proven to significantly increase conversion. WEVO has successful experience working with Wells Fargo, Fidelity Investments, Blue Cross Blue Shield, Harvard University, Intuit and others.