The Ultimate Guide To A/B Testing

A/B testing (sometimes called split testing) is comparing two versions of the same web page to see which one performs better than the other in achieving the goal. You compare two web pages by showing the two variants (let's call them A and B) to similar visitors at the same time. The one that gives a better conversion rate, wins!

All applications have a goal to achieve:

• SaaS web apps want visitors signing up for a trial and converting to paid visitors
• eCommerce websites want visitors buying products(conversions)
• News and media websites want readers to click on ads or sign up for paid subscriptions

Every business website wants visitors converting from just visitors to something else. This process is called conversion. The rate at which a website is able to do this is its "conversion rate". Measuring the performance of a variation (A or B) means measuring the rate at which it converts visitors to goal achievers.

Why Should You A/B Test?

A/B testing allows you to make more out of your existing traffic by optimizing the whole conversion process. While the cost of acquiring paid traffic can be huge, the cost of increasing your conversions is minimal. To compare, a Small Business Plan of Visual Website Optimizer starts at $49. That's the cost of 5 to 10 Google Adwords clicks. The Return On Investment of A/B testing can be massive, as even small changes on a landing page or website can result in significant increases in leads generated, sales and revenue.

What Can You Test?

Almost anything on your website that affects visitor behavior can be A/B tested.

1. Headlines
2. Sub-headlines
3. Advertisements
4. Paragraph Text
5. Testimonials
6. Call to Action text
7. Call to Action Button
8. Links
9. Images
10. Content near the fold
11. Media mentions
12. Awards and badges

Advanced tests can also test different pricing plans, data projections, sales promotions, free trial periods, navigation and UX experiences, free or paid delivery, and more.

Step-by-Step A/B Testing Process

The correct way to run an A/B testing experiment is to follow a step-by-step process. It includes the following steps:

1. Study the Data: Use any of the website analytics tools as Google Analytics, Mixpanel, Heap Analytics and find the problem areas in your conversion funnel. For example, you can identify the pages with the highest bounce rate. Let's say, your Product page has an unusually high bounce rate.
2. Observe User Behavior: Utilize visitor behavior analysis tools such as Heatmaps, Visitor Recordings, Form Analysis and On-page Surveys, and find out areas of problem. For example, “The Call-To-Action button is not prominent on the product page."
3. Construct a Hypothesis: Per the insights from visitor behavior analysis tools, build a hypothesis aimed at increasing conversions. For example, “Increasing the size of the Call-To-Action button will make it more prominent and will increase conversions.”
4. Test your Hypothesis: Create a variation per your hypothesis, and A/B test it on the original page. For example, “A/B test your original product page against a version that has a larger CTA button.” Calculate the test duration with respect to the number of your monthly visitors, current conversion rate, and the expected change in the conversion rate. (Use our Bayesian Calculator here.)
5. Analyze Test Data: Analyze the A/B test results, and see which variation delivered the highest conversions. If there is a clear winner among the variations, go ahead with its implementation. If the test remains inconclusive, go back to step number three and rework your hypothesis, implement.
6. Report the results: Let others in Marketing, IT, and UI/UX know of the test results and the insights generated and discuss the changes for better conversions.

Image credits: www.netlifyusercontent.com