How to Implement Incremental Testing in Marketing

For a business operating in today’s world, not being data-driven is not an option. Measuring the incremental impact of your paid channels in marketing is more important than ever. You’ve probably heard about running GeoLift experiments as the gold standard for this purpose. Perhaps you’ve even implemented some incremental tests in-house. But are you confident that you executed them correctly? Poorly designed and executed incremental testing can harm your business rather than benefit it, leading to misguided strategies and wasted resources.

In this article, we will explore the essentials of properly designing an incrementality test, look under the hood explaining how things are working, and discuss the critical steps to take after testing. Whether you’re a seasoned marketer or just getting started with incremental measurement, this guide will provide valuable insights to ensure your tests drive meaningful and accurate results.

What Are GeoLift Experiments?

A GeoLift experiment applies the logic of an incrementality test at a geographical level. This approach compares regions exposed to the campaign (treatment) with similar regions not exposed (control), isolating the true effect of the marketing activity. By focusing on geographic granularity, GeoLift ensures that macroeconomic shocks are represented equally in both the test and control groups, truly isolating the campaign’s impact.

What Is Happening Under the Hood?

In GeoLift tests, a suitable control group (or “counterfactual”) is created using a synthetic control method to closely mimic the characteristics of the treatment group. This involves selecting regions similar to the treatment group in terms of demographics, market conditions, and historical performance. Statistical models, such as matching algorithms and regression analysis, are employed to ensure the control group closely resembles the treatment group before the campaign starts.

The modeling occurs in the pre-treatment period, allowing for the evaluation of how well the synthetic control group matches the actual values before launching the campaign. This process also helps identify the minimal uplift needed to achieve statistically significant results, ensuring the testing is sophisticated and precise. By constructing this synthetic control group, the true effect of the ad treatment becomes clearly visible during the treatment period, as any observed differences in outcomes can be attributed to the marketing campaign rather than external factors.

This method enables the accurate measurement of the effectiveness of paid channels and the true value of marketing efforts. Additionally, both treatment and control groups account for external shocks during the treatment period, ensuring that the observed impact is genuinely due to the ad treatment. This robust approach provides reliable, actionable insights into the incremental impact of marketing activities, facilitating better strategic decisions and optimized budget allocations.

What Comes After Testing?

Great, you have empirically proven whether a specific channel or channel mix yields incremental results. You now know if the attributed metrics reported by a given platform are accurate. But what’s next? You don’t want these valuable test results to get lost in spreadsheets or forgotten in presentations.

Fortunately, beside incrementality tests, another powerful statistical model comes in handy: Marketing Mix Models (MMM). MMMs provide a holistic analytical overview of how each channel or campaign influences your target KPIs (e.g., sales).

Furthermore, MMMs can incorporate the results of your incrementality tests, increasing their predictive accuracy with each test. Over time, this integration builds a robust tool fueled by causality, allowing you to steer your marketing operations in the most efficient, forward-looking way.

Interested in designing and running your first incrementality test or understanding how to combine incrementality tests with Marketing Mix Models? Feel free to reach out to our team to get started.