How A/B testing a simplified email signup popup boosted conversions on Newspack sites

By Teresa Ristow

Overview

When building email signup popups on the Newspack platform, publishers can include options for readers to sign in with Google or log into an existing account through the modal. A simplified approach where a reader’s only choice is to enter an email and submit the form is also available. A/B testing done late last year by BlueLena reveals a notable bump in the signup rate for publishers choosing the simpler approach.?

Top-of-funnel growth is a top priority for news organizations, who are continually looking for ways to grow the number of contacts in their email database. Among the most effective tactics for gathering new email signups is through the use of on-site prompts, including popups that ask readers to sign up for newsletters and email updates. When evaluating the performance of these popups for publishers utilizing the Newspack platform earlier this year, BlueLena saw an opportunity to A/B test aspects of the user experience in hopes of increasing the conversion rate and generating more signups.?

BlueLena recorded the performance of existing newsletter signup popups on the websites of five publishers, with the publications averaging a 0.9% conversion rate on the popups — of all of the times the popup appeared on the website for a reader, it was completed 0.9% of the time. In each case, the publisher was utilizing a popup that asked for a reader’s email, offered an option to sign in with Google, or offered a “log-in” option for readers who may have signed up in the past.?

BlueLena hypothesized that the variety of options was hindering signups by offering readers too many choices, and that giving readers only one option — to enter their email and hit the “Sign up” button, would lead to a higher email signup conversion rate.?

For the test, BlueLena disabled the current popup campaign and enabled a new version which removed the “Sign in with Google” option and the “Already have an account? Sign in.” option.?

The new popup was tested for a minimum of three weeks for each publisher before comparing conversion rates before and after. All five publishers who utilized the simplified popup saw improved conversion rates during the testing period, with the average conversion rate increasing from 0.9% to 1.45% over the testing period.?

While an A/B test that showed the popup variants during the same time period wasn’t possible, it was possible to run each variant for a set period of time and then compare the results. This type of testing is useful when comparing options on a platform that may not have A/B testing capabilities built in.?

About BlueLena?

BlueLena creates a sustainable future for independent local media with strategies to build audiences and tools to support journalism funding models for long-term success. Today, over 200 brands leverage our audience management platform, strategy consulting and marketing services to accelerate their digital transformation.

BlueLena is founder-funded and backed by investment from Automattic (owner of Newspack), Local Media Association and Old Town Media, a New York-based firm that has supported The Colorado Sun, Block Club Chicago, Chalkbeat and Medium, among other successful media-related projects. Visit us at bluelena.io?