Concrete (Hard) Questions You Should Ask Your Verification Vendor

I was told I was not concrete enough when imploring advertisers to ask harder questions of their fraud and brand safety detection vendors, after they were exposed for not doing their job -- 1)?Ads Still Funding Russian Disinfo , 2)?Domain and Page Spoofing Remains Easy and Rampant , 3)?We’ve Known Brand Safety Tech Was Bad—Here’s How Badly It Defunds The News , 4)?Myspace Got Caught Up In A Massive Ad Fraud Scheme

So here are?concrete questions?you can ask,?concrete data?you should get, and?concrete actions?you should take to check if you are getting what you thought you paid or, or what they promised you "in communications using an interstate communications device (internet)."

Ask them to show proof (not excel spreadsheets)

Why was something marked IVT ("invalid traffic") or fraud? Explain in simple words not technical jargon why something was marked fraud -- like the 1% IVT they show in their dashboard and excel reports. More importantly have them show proof for the other 99% that they didn't mark as IVT. Since DoubleVerify wrote in their own press release about the "well-documented occurrence of verification stripping ... removing verification tags from the ad being delivered" ask them HOW they detected anything if their detection tags were stripped? If they can't answer or won't answer, that tells you something.

Ask them to explain the discrepancy between vendor 1 and vendor 2

I have reviewed dozens of cases over the years where the fraud detection vendor used by the advertiser reports there is high IVT, so they ask for a refund. But the fraud detection vendor used by the publisher or seller shows low IVT, so they refuse to give the make-good or refund. Each vendor just provided a number like 10% IVT versus 1% IVT. Neither can explain how they measured the IVT; neither can explain why their measurement is different from the other vendor when measuring the same campaign during the same timeframe; and neither will give you any details so you can understand what is going on and troubleshoot. Tell the vendors to work it out amongst themselves and provide you with a written document after the investigation to explain the discrepancy. Withhold payment or threaten to not renew the fraud verification contract until they complete the investigation.

Ask them to explain the discrepancy between vendor 1 and vendor 1 (not typo)

There's even been cases where the same vendor measuring the same campaign comes up with different IVT numbers. The vendor was used by the publisher, so their tag was on the page. The same vendor sold their IVT detection services to an advertiser, and the tag is placed in the ad, and delivered via the ad iframe. Did the vendor differentiate the tag? After all on-site measurement is different than in-ad measurement and the tags should be tuned differently. If they are using the same tag for both -- on-site for the publisher, and in-ad for the advertiser -- and they don't realize that is an important distinction, that is a problem. The fact that they didn't know tells you something.

Get direct access to dashboards, so you can check the data with common sense

If you don't have access. Get access. You are the customer paying for the fraud detection. Why would you not have access to the dashboard. When you have access, you can sooner see if there are discrepancies you should ask about. For example, you purchased 10 million ad impressions, but the fraud verification dashboard shows only 100,000 impressions measured in the same time period. Why? I've reviewed cases where fraud verification vendors sampled the inventory to make more margin without telling clients. Of course, this could also have been accidental. If you have access to dashboards, and you chose to look, you would have been able to spot the "oopsies" more quickly, not the next year.

Ask what was measured exactly and how much was paid

You should also get access to and review the contract and fees paid by the agency to the fraud vendor. There have been cases where agencies marked up the fraud verification services without telling their own clients, just like there was undisclosed markups and arbitrage of digital media by agencies, as uncovered by the 2016 ANA Transparency Investigation. We haven't even mentioned kickbacks to themselves hidden through foreign subsidiaries, or kickbacks from fraud vendors for exclusivity and to push their detection services on clients.

Check if you are double- or triple-paying for fraud verification

How is it possible that these greedy bastards are double- and triple-dipping to make money off of the same campaign when their tech is suspect at best? Easy. When the ad exchange is paying for fraud detection, the CPM+ markup is bundled -- i.e. hidden -- with the cost of the media. You paid $3 CPMs thinking that's the media cost. You actually got $2.85 CPM media with a $0.15 CPM tacked on for fraud verification, without you realizing. This does not show up as a separate line item because it's comingled with the media CPM. You might have a separate line for fraud verification. If you paid that, then you are paying twice for fraud detection on the same campaign. This assumes the agency wasn't doing shady stuff and marking up the fraud detection fees or getting an undisclosed kickback from the vendor for locking you in to a three contract with that vendor (they sold you on the discounts you would get).

Ask if your agency is doing a lot of unnecessary busybody work

Why should you ask this? I've witnessed campaigns run by media agencies that had 1,500 different ad creatives. The fraud verification vendor supplied 1,500 different verification tags, one for each ad creative. If you ponder this for a second, you will realize that that is completely unnecessary and useless -- to have fraud detection by ad creative. The bots don't look at the ad or care about what the creative message is. Their job s just to cause the ad to load, any and all of them, so they can make money fraudulently. Having different tags for different creatives is entirely unnecessary and creates a large amount of busywork which you are paying for. The media agency loves this because they do accounting based on billable hours (even if they have a fixed price contract with you; they need to burn hours). One fraud verification tag is all that is needed, and it should be placed at the campaign level, so all 1,500 ads have detection. That's how we measure campaigns with an in-ad FouAnalytics tag.

Ask what do you should do with a number (like X% IVT)?

If the fraud vendor doesn't tell you which sites and apps were causing the fraud and eating up your ad impressions, you can't troubleshoot the campaign. That's exactly how they like it -- because you have to keep paying them for fraud detection. If you could see which sites and apps were the most egregious cheaters, you can add them to block lists or remove them from include lists while the campaign is still running. That way you can clean your campaigns and make them better while they are in flight. The more money you prevent going to fraudsters, the money money is left over for ads on better sites and apps.

Ask them when was the last time they updated their detection algorithm?

I look at FouAnalytics data every day and add and edit rules in our detection algorithm every week. For example, the recent adjustments included adding real Roku device identifiers and version numbers to the detection. Ask the IVT detection vendor when was the last time they updated their detection algorithm. More importantly how do the engineers that work on the back end code learn about new attacks and obfuscation methods. Do they interact with customers and review data, or is that not their job (i.e. handled by a different department)? If the coders to modify the algorithm don't interact with customers, review threat data and study new scenarios, how could their detection possibly keep up with the bad guys' clever tactics? And ask them, if there was an emergency situation like the Russia-Ukraine war, how quickly can they get a code change approved and rolled out into production -- e.g. to detect ads going to Russian disinformation sites and stop those? Probably not fast enough.

Ask what portion is of the detection is marked as "unknown"?

In addition to "unmeasurable" (where their detection tag was stripped out), what portion of the portion they did measure was marked as "unknown?" The spreadsheet on the right is a summary from a fraud vendor for a live campaign. The largest row at the top is marked as "mobile in-app" -- it appears that all apps are dumped in the same bucket, and all of it appears to be marked as "fraud free." How do you tell the fraudulent ones apart from the legit ones if they are all mixed together like that? You can't. Very unhelpful and useless. That bucket is about 29% of the total at the bottom. So if 1/3 of your ads went to apps you can't see, how can they mark it as fraud or not? Do they just assume that "it's fine" and NOT IVT? Perhaps that's why they consistently only find 1% IVT. Do YOU assume that "unknown" is fine? If the verification vendor can't tell you where your ad ran or have no idea if your ad even ran or not? That should tell you something, right?

I've got a lot more questions, but I've bored you long enough. ;-)

Think of other questions you can ask. Practitioners, if I have missed some juicy examples, please let me know. This is not new stuff, I have previously published examples -- e.g. B2C Marketers' Anti-Fraud Playbook. Maybe today someone will take a closer look, ask harder questions, and stand up for themselves so they don't get ripped off more. Let me know. I can help (it can be entirely behind the scenes).