Annual Reminder that Correlation is not Causation

Just because sales were happening while you were doing digital marketing doesn’t mean those sales were caused by the digital marketing you were doing. If you’re not actively measuring incrementality — i.e. did your marketing activity drive any incremental sales that would not have occurred without the marketing? -- then you’re looking at correlation at best, and you’re very likely getting ripped off too. The sales you saw were CORRELATED in time to the digital marketing you were doing, not CAUSED by it.?

?One of the funniest ways to illustrate correlation vs causation comes from Tyler Vigen’s Spurious Correlations website. Did “Nicholas Cage appearing in more films” cause more people to “drown by falling into a pool?” Of course not, but those things were correlated in time over a period of 10 years! Correlation is not causation.?

?I was asked recently if the “performance” seen in digital marketing over the last decade was all fake if much of the analytics and attribution were faked. That’s where the concept of correlation vs causation comes in handy. There were real sales, but they were not necessarily caused by the digital marketing. The sales of soup, soda, and soap, by some of the largest CPG companies correlated to the large spending they were doing in digital; but most or all of those sales would have happened anyway. This was clear when P&G turned off $200 million in digital ad spend and saw no change in sales. The sales happened anyway, without the digital advertising. In a very few select cases, the marketers were doing good incrementality measurement. But in most cases the biggest brands were not doing any incrementality measurement; in fact, many of the largest marketers claimed they were “just doing branding” with their digital ads and didn’t make the effort to connect that with the sales that occurred in offline stores. It blew me away that they didn’t even know what incremenatility was, let alone how to calculate it. There are a number of known DTC (“direct to consumer”) success stories, however, where new brands launched entirely in digital and could track the direct causation of sales by the digital marketing they were doing. Large brand advertisers’ digital marketing campaigns were the opposite of that.?

?Digital marketing throws off a lot of data. But just because you can measure it doesn’t mean it’s right, good, or useful. Your ads shown on millions of long-tail sites (“scale”) doesn’t mean they are useful, since most of the traffic on those long tail sites are not humans. You know that showing ads to bots won’t produce any business outcomes right? Chase confirmed this when they reduced their programmatic reach from 400,000 sites showing their ads to just 5,000 sites (a 99% decrease) and saw no change in business outcomes. Further, you know that just because the fraud detection tech companies you paid for can’t detect the bots, it doesn’t mean there are no bots. It just means they couldn’t detect them. I challenge you to use FouAnalytics to take a closer look. Or are you afraid to find out??

?Just because you paid lower CPM prices for the ads (“cost efficiency”), doesn’t mean you got a better deal, if most of your dollar went into the pockets of the ad tech middlemen instead of toward showing ads and your ads were shown on crappy long tail sites that had very few human visitors. Those sites can afford to sell ads at low CPMs because unlike real publishers they don't have any costs of content -- those long tail sites just plagiarized all the content or generated it with algorithms. I challenge you to get a list of domains your ads were shown on and run PageXrays on them to see how crappy they are. Vast majority of the sites are auto-generated using wordpress templates and stuffed to the gills with ads, not to mention other shenanigans like stacked ads, pixel stuffing, pop-unders/ups, off-page ads, auto-refreshing, self-clicking ads, etc.

?Just because you got more clicks (“performance”) doesn’t mean your ads worked, or that humans saw your ads and clicked on them. You know that bots can click on ads and do so deliberately to create the illusion or “performance.” This tricks the buying algorithms to allocate more budget to these sites because they were told to optimize for higher CTRs (“click through rates”). I challenge you to get a domain level placement report with corresponding CTRs for each domain. Look for domains that have greater than 10% CTRs and check them manually. If they look like “frappy” (fraudulent + crappy) sites, add them to your block list.?

?To wrap up this year’s installment of “correlation is not causation” I can confirm there were real sales over the last decade of digital marketing. Only some of those sales were caused by the digital marketing. Most of those sales were simply CORRELATED to digital marketing. Just having a lot of data and metrics still doesn’t mean the digital marketing caused the sales, especially if those analytics were not measuring incrementality. Quantity metrics like the number of impressions, amount of traffic, and number of clicks were easy to measure and easy to report. More impressions, more traffic, and more clicks did not necessarily cause sales. But they certainly tricked algorithms to allocate more budget to it. Further, loss of device identifiers due to Apple’s ATT (“app tracking transparency”) and the loss of third party (“3P”) cookies due to Google Chrome’s deprecation of 3P cookies don’t mean the loss of sales. Most of those sales would have occurred anyway. Loss of device identifiers and 3P cookies means the analytics and attribution systems built on those won’t be able to do reporting as they used to. But marketers never needed to know which individual device bought the soup, soda, or soap, or installed an app to know if their marketing worked or not. TV, print, radio, and billboard advertising all worked and and continue to work without device level analytics and attribution.

?As a marketer, be sure to look beyond the vanity metrics and quantity metrics that digital marketing produces to put in place mechanism to measure true incrementality of sales -- what sales were caused by the digital marketing above the sales that would have happened anyway in the absence of the digital marketing. This way it helps you avoid the outright fraud of analytics and attribution manipulation that fraudsters have two decades of practice doing.?

?Here's how "performance" is faked too

https://www.forbes.com/sites/augustinefou/2020/10/30/gaping-security-loopholes---why-marketers-should-care/