Bid duplication - How to check using FouAnalytics

In this week's article -- Programmatic’s Hidden Scourge Of Bid Duplication -- Sarah Sluis articulated the problem of bid duplication succinctly: "When bid requests go out through different SSPs, they are indistinguishable from each other – despite the existence of an ID to address this issue. Buyers sift through dozens of duplicate bid requests, unable to tell the difference between them. Twenty bid requests from an SSP for one ad may look identical to 20 SSP bid requests for 20 different ads." The problem this creates for ad buyers is that if their DSP's algorithms are not fast enough to DE-duplicate the bid requests, the advertiser ends up bidding against themselves many times over and therefore ends up paying more for the same ad opportunity.

Bid duplication: gray area or deliberate fraud?

Most people consider the problem of bid duplication to be a "gray area." After all, SSP's algorithms were programmed to do this as "yield optimization" supposedly to help their publishers earn more money. Not only does this earn the ad seller higher CPMs per ad unit, it also increases the number of bids they receive and the probability of getting a higher winning bid. But just like any tool or technique can be used for good, this technique can also be abused by bad guys to cheat.

For years, fraudulent publishers have sent large numbers of duplicate bid requests deliberately through as many SSPs as possible. They know various DSPs are not able to de-duplicate the bid requests fast enough, so they get many more bids on the same ad opportunity than if they didn't cheat. This was a sure-fire way to get many more bids and higher winning bids. The sample data below was published by DeepSee.io - Publisher Intelligence three years ago - https://deepsee.io/blog/header-bidding-benchmarks-2021 showing 80 duplicate bid requests for a single ad slot.

How to check for bid duplication using FouAnalytics

If you use FouAnalytics in-ad tags for measuring your programmatic ads, you will find a data grid called custom_AUCTION_ID in the dashboard. Note the 4 examples below -- Trade Desk, Xandr, Basis, Yahoo DSP. The formats of the auctionIDs (unique identifiers for bid requests) are slightly different for each DSP, but note the counts to the left of each auctionID. If those numbers are significantly more than 1, you have bid duplication. Note the Xandr example specifically, where we see the same auctionID being sent 47, 30, 29, 24, etc. times.

Taking a closer look at the other data grids in FouAnalytics for the Xandr example, we see what could be "errors" but may also be attempted fraud. The same auction ID was sent 47 times over the course of about 530 milliseconds -- 1705725118 to 1705725647. What's also funny in this example is that it's a video ad (OLV - online video) and the fraudster also sent faked video completion events. Notice 16 ads were marked as "25th_quartile" and another 14 were marked as "complete." For more background on how easy it is for fraudsters to falsify video completion rates -- Falsified video completion rates, easy.

Finally, we observe that bid duplication happens more often on certain supply sources (e.g. SSPs engaged in aggressive "yield optimization"). Bid duplication is also highly correlated with fraudulent sites and mobile apps. Fraudsters have also specifically targeted OLV (online video ads) and CTV ads because those earn them far higher CPMs for the same amount of computing power (to deliberately send duplicate bid requests). Notice the color coding under each auctionID below.

If this is helpful, please let me know. If others need to see this, please share with them. As always, message me directly if I can answer any questions or be helpful to you. Happy Saturday Y'all!

Further reading: https://www.dhirubhai.net/in/augustinefou/recent-activity/newsletter/