The Origin of A/B Testing

A/B testing is at the heart of the bustling start-up ecosystem. It is a contemporary catch-phrase that you will hear over, and over, and over again among entrepreneurs, techies, and VCs as often as you hear about Fintech, Series A funding, and Big Data.

However, this modern concept actually dates back to a completely different and non-digital age, and it is almost 300 years old. The origins of A/B testing can be traced back to James Lind’s 1753 A Treatise of the Scurvy. Scurvy was the leading cause of disease and death among seamen in the 18th century. James Lind’s clinical trial showed that citrus fruit was beneficial against scurvy, whereas other remedies had little effect. Unfortunately, it took the Royal Navy 40 years to issue citrus juices to seamen.

James Lind’s well-planned clinical trials provided supplements to the otherwise identical diets of 6 pairs of men with symptoms of scurvy. The 6 pairs were kept in the same sickbay and given the same care. 4 pairs showed no improvement, while the pair that received a quart of cider each day improved slightly. However, the two sailors that were given 1 lemon and 2 oranges daily made astonishing recoveries.

Today we know that scurvy is caused by a lack of vitamin C. This is why citrus fruit with their high vitamin C content are effective at preventing and treating scurvy.

Much like James Lind’s 18th century search for the cause of the scurvy and its remedy, A/B testing is a controlled experiment. You set up two scenarios that are identical, except for one feature. This allows you to test which of the two options work better at catching the attention of your audience, engaging them to take action, and ultimately converting eyeballs into $$$ for your bottom line.

A team at Google couldn’t decide between two blues so they tested 41 shades between each blue, showing each shade to 1% of their users to see which one performs better.

Google famously tested 41 different shades of blue to decide which color to use for advertisement links in Gmail. The company showed each shade of blue to 1% of users. A slightly purple shade of blue maximized the likelihood of users clicking on the advertised links, giving Google a tangible $200m boost in ad revenue.

A Primer on A/B Testing:

The A/B Testing Checklist You'll Want to Bookmark

Sources:

A Treatise of the Scurvy, Original Text

The Guardian: Why Google has 200m reasons to put engineers over designers

Credits:

• University of Virginia
• Google A/B Testing Example