Touchstone identifies a 16% improvement in Email Open Rate

Subject Line Test Case Study

This is a run through of an A/B Test conducted by Alchemy Worx for one of our clients 2 weeks ago. We wanted to test a Teaser Subject Line (SL) for their discounted parking offer against an Explicit SL that spelled out what the offer was.

Subject Line A (Teaser): A new perk for our customers

Subject Line B (Explicit) ???? Airport Parking Deals and Discount Code

Fig1

• As You can see in Fig1 Touchstone predicted that SL A would perform poorly - Open Rate 22% below average, Click Rate 2% Below average and a delivery rate 6% worse than average. However although SL B performed much better than A - Open Rate 2% below average. Click Rate 1 % above average plus a better than average delivery rate, the results are still not that great.

Had this been a live A/B test where real emails were sent out to a percentage of their list – real customers – not only would they have exposed a significant number to the poor SL and risked their reputation in terms of deliverability. We would have had no option but to send out B which turned out to be a very average SL to the rest of their list.

What you can also see in Fig1 is Touchstone also identified words within the SL that could be improved by highlighting them in Red (When using Touchstone, you can see the recommendations by hovering your cursor over the word). 

Using this feature we tweaked the two SLs as follows and re ran the test.

Subject Line C - The Latest Reward for our Members

Subject Line D - ???? Airport Parking Offers and Discount Code

 Fig 2

This test resulted in a massive improvement in the predicted performance of both SLs

Subject Line C has a predicted improvement in Open rates of +36% above average and +16% in the case of Clicks. While SL D ended up the overall winner with predicted Open and Click Rates of 16% and 15% above average respectively. So Subject Line D went out to the entire list and no money was left on the table. Total time spent 15 minutes.

On average Touchstone users achieve a 14% lift in Open Rates.

Touchstone - Subject Line Testing Made Easy.