How to check if you got what you paid for in programmatic?

When buying through programmatic channels, you know there is waste and fraud. But did you know how much fraud and waste there is? Some people jokingly quote Wannamaker -- "50% of my ad spend is wasted ... " I don't joke around. There's years of evidence that shows that at best, 50 cents of your dollar goes to publishers for showing ads; the rest goes to middlemen, toll-takers. The reality of the fraud and waste is far worse, but no one is ready to admit that because they really really want to believe that digital works. Here are two articles for you to consider.

Feb 2, 2022 -- Programmatic display ad productivity is 1%, using round numbers

Feb 2, 2022 -- Why does digital appear to perform so well, and fraud appear so low?

This article shows you one of the ways I've helped clients check for waste and fraud in programmatic channels over the years, without any specialized tech.

Bids won versus impressions served should be 1-to-1

In programmatic, the basic assumption is that when you win a bid, you get to serve your ad. So there should be a 1-to-1 relationship between bids won and impressions served. Over the years, clients have asked for more detailed reporting from their DSPs. Instead of monthly totals and averages, we asked for breakdowns by domain and app -- so we can see the exact quantities of bids won by domain or app. Notwithstanding the errors I see in those reports -- e.g. "not set" is not a domain or app -- we can use the numbers from these detailed reports to compare to the numbers we get from the ad server. Again, you have to ask for detailed reporting, breaking down the quantities by domain and app. Monthly summaries don't reveal anything; and you won't be able to tell if anything is wrong.

In the slide above you see two examples of this analysis. When we compare bids won to impressions served, by domain, for a specific time period, we can see some domains have only small discrepancies. For example, weather.com and spotify (mainstream publishers that you've heard of) have single digit discrepancies. These are acceptable tolerances. You should know that ad tech is so bad and inaccurate that single-digit percentages of discrepancies is already really good. As we look down the column, you can see the discrepancy between bids won and ads served to continue increasing, to the point that in the last row, you see a 100% drop-off. That means we could not record any ads served, for the bids won.

The greater the discrepancy, the more likely something is shady

A general rule of thumb to take away is that the greater the discrepancy the more likely the site or domain is shady or fraudulent. In any case, if you see discrepancies greater than 10%, you should probably investigate more deeply. After all, you paid for the bids won, but didn't get your ad served. Note in the second example on the right side of the slide, we looked at a low CPM campaign. Going in, we knew to expect more shady sites and apps in the mix. And the data we saw supports this -- the discrepancies were larger and the overall average was -74% (a 74% drop off, which means only 26% of the impressions were served, compared to the bids won). From my research, the more shady sites use more bot traffic. And you may remember from this article (Tools and Techniques of the Ad Fraud Trade ), bad guys are very optimized. If the bot doesn't need to load the entire webpage, it can same time and bandwidth by loading just the ad impression to make money; if they don't need to load and render the ad, why wait around for the ad to serve, arrive, and render? This is what is happening -- the bots win the bid and they move on to the next task without waiting for the ad server to actually serve the ad. This also happens in mobile when the ad serving process is too slow and the user has already scrolled past the ad slot.

For those of you who don't have access to ad server reporting, you can do the same measurement by having a FouAnalytics tag in your ad impressions. The in-ad tag of FouAnalytics executes after the ad is rendered in the browser/device. So we know the ad arrived and was rendered on screen. When you compare the quantities from FouAnalytics, by domain or app, with the quantities reported by your DSP, by domain or app, you can check for the discrepancies mentioned above - the drop off between bids won and ads served. Look for anything that is higher than 10% and start asking more questions. You can also start to add suspicious domains and apps to your block list to avoid those that have high discrepancies.

A more advanced example where we serve the ads too

For some clients, over the last 10 years, I have also operated the ad server (Revive) that serves their ads. This was not only to help them save costs, but also to get an additional detailed data set to help us assess the accuracy of the whole process -- from bids won, to ads served, and finally to ads rendered. The chart below shows an example of this. At the DSP level, we purchased 1 million ad impressions. Our ad server showed us that 931k impressions were served - a 7% drop off. And finally, the FouAnalytics tag at the bottom shows 746k ads rendered (the FouAnalytics tag fired).

In this example, we paid for 1 million bids won, but we could only record 75% of the ads rendered on screen. In other words we didn't get 25% of what we paid for. There are two ways to reduce this form of waste -- 1) by trying to get a refund from the DSP for the 25% but they will challenge that because they don't want to give you the money back and in their eyes, they are only responsible for the bids won; 2) add obviously bad domains and apps to your block list, or remove them from your allow lists so you can clean your campaign and avoid the shady sites where large discrepancies tend to occur.

Please let me know if you run this analysis yourself. I would be interested to see what you find in your own campaigns. If you need a FouAnalytics in-ad tag so you can see the "ads served" in your campaigns, please let me know. I will leave you with one last example of my experimental campaign -- 814k bids won, 549k ads served (according to my ad server), and 546k ads rendered (in-ad tag). In this case, the large drop of -33% came from the ad server level. Once the ad was served, most of it arrived and rendered (546k out of 549k).

P.S. Note that we haven't even talked about the "drop offs" between the clicks reported by your ad platform and the clicks that actually arrived on your site, according to your site analytics. It's worth taking a closer look with FouAnalytics - you can use the on-site code on your sites (you can copy and paste into Google Tag Manager). It will help to troubleshoot and explain the discrepancies you see in Google Analytics. GA filters some bots, but it doesn't tell you which bots they were and how much there was; FouAnalytics does. See this article for reference: https://www.dhirubhai.net/pulse/marketers-dont-optimize-your-campaign-performance-dr-augustine-fou/