Every model is wrong, but you can make it “more right”

Sebastian Vogg Founder & Performance Marketing Manager at VOGGSMEDIA and? Nicolas Arrive , Manager, Marketing Science, Meta

German version available @ https://www.dhirubhai.net/pulse/jedes-modell-ist-falsch-aber-du-kannst-es-richtig-machen-vogg/

What has happened to the future of performance advertising?

Fundamental changes to the digital ads ecosystem have impacted profoundly performance marketers. Cookies and personal identifiers used to store consumers’ digital footprint across the web and wired Attributions. Google’s Chrome made it clear (see update in April 2021) that cookies are going away.

So too did Apple by rolling out the? App Tracking Transparency framework. Regulations aiming at further protecting consumers' privacy are now alive across many geographies (GDPR and ePrivacy regulations in Europe, CCPA in California, Brazil General Data Protection Law). Deterministic user-level measurement of digital advertising is no longer a reality.?

Attribution tools are getting less and less accurate simply because the data is not there. More precisely, consumers’ journey touch poiunts are no longer tied to each other and to an individual. Reversely Marketing Mix Models (MMM) are back in fashion. Privacy conscious MMM are statistical models that ingest many variables that could impact sales, including paid media variables. Through a process of complex statistical analysis it figures out which factor or marketing channel drives sales. The release of Open Source solutions by Google (Lightweight MMM) and Meta (Project Robyn) have fueled wider spread usage, reducing barriers to entry. Clients still need to invest some resources (data feed and the right skills to run these models) to get a useful MMM to assess their marketing performance.

Who does what?

MMM are by definition produced modeled results. Meta does not hide that it models conversion in Ads Manager (see here). Google as well (see here). Google’s transition to GA4 moved all clients away from deterministic attributed conversion count to modeled conversions. Google mentioned very recently here, that starting in June 2023, it’ll remove first click, linear, time decay, and position-based attribution models and will rely on data-driven attribution, another word for modelling what the conversions count should be.

If reporting tools model, if Attribution tools model, and the new modern MMM in vogue do also model , how do you know who is right?

Ad experiments remain very important. These can take the shape of geo-experiment or in-platform experiment (i.e conversion Lift - see here or here). Randomized groups of consumers are exposed to ads, a statistically similar cohort is not. The difference in sales or brand metrics observed between the two groups is the true incremental impact of ads. Experiments, if well designed, can represent the ground truth, but these are not always-on.

Understanding potential bias from media platforms is not achievable, but minimising or understanding how far off these models are, and that is what we do at VOGGSMEDIA , and we rely on Experimentation.

What do we do at VOGGSMEDIA and why?

Using experiment based results to bring your Performance marketing report closer to the true contribution of one or more channels to our clients business is possible.?The topic of attribution is one that is very present in our day to day work with our clients.

No matter which attribution tool one uses, you always find yourself at the same crossroads: you have to believe in the accuracy of the tool or platform and the data underneath both, and you can never be 100% certain that you really know how far you are from the truth.

VOGGSMEDIA is a big believer in Experimentation on Meta with Conversion Lift studies for instance, in order to get to know the overall on-platform "picture" which have shown themselves to be very accurate. Comparing these findings with our numbers in Ads Manager and in Google Analytics gives us a great framework to understand what drives incremental performance, and how much truly incremental gains Voggs' clients are getting.

We hold a channel true to the incremental number of conversions we obtain from experiments. We then redistribute the remaining conversions across other channels. The total all-up conversions or sales count is unchanged. Experiments pin down the share of incremental conversions from one, hopefully two channels. In times of iOS14, more privacy and less tracking ability than ever, since we do have relevant data and shouldn't undermine the validity of data gathered from experimentation. Making good use of these experiment base data points help us paint a more accurate picture of the causal attribution and contribution situation.?

As we have learned in the past, many attribution tools are just as much (or even more) guesswork. Depending on your attribution model, you can always make a case for increasing or decreasing any of the platforms. But in the end you will end up at the same conundrum as before: Which numbers are actually correct and who is truly causing my conversions to happen incrementally and from where?

So yes, every model is wrong and that’s why you have to find the right path. And Experimentation is a solid foundation in your search for data truth.

Thanks to Paul Fagan and Matthias Vogg for their though provoking ideas. Always good to be inspired by Jin Choi , Igor Skokan and many others.

This was co-authored with Sebastian Vogg

#effectiveness #cookieless #experiments #measurement #ads