How do advertisers identify good publishers?

What do you look for to identify a good publisher? Is it that their sites have low to no bots and lots of humans? That's a good place to start. But it's a bit more nuanced than that. Let me show you.

The simplest cases

In the example on the left you see lots of dark blue (humans) and very little dark red (bots). That's a good example of a site with lots of humans and few bots. On the right side, you see an example of lots of bots (dark red) and virtually no humans. But would you believe these are relatively rare in the real world? Very few sites have that much humans and so few bots (left); and very few sites have that much dark red bots (that is an example of an amateur bot maker that didn't disguise their bots well at all).

Why have you never seen these before? These charts are from FouAnalytics. Google Analytics and Adobe Analytics do not label humans or bots. The fraud verification tech that you may also use only reports the percentage IVT ("invalid traffic"); they don't measure for humans either. When they report 1% IVT, you should not assume the other 99% is human. In fact you should assume that for that 99% they simply failed to detect anything wrong or their detection tags were blocked or stripped out by bots trying to avoid detection [1 ], [2 ], which means they have no data; not that there is no fraud.

The above are the simplest cases, easy to understand. But they are not that common.

More common scenarios

Is the FouAnalytics chart above on the left showing a good publisher or not? Make a guess first, before reading on. This is a good publisher's site that I have measured for more than 5 years (and have not charged them anything). As you can see, a good publisher's site might have bots too. They don't want the bots to come to their site; they didn't buy traffic. But their site is publicly accessible on the web; so any bot can visit their webpages. If the publisher has valuable content, scraper bots come to steal their content. The chart on the right shows exactly this example. Notice the large spikes in red. Those are "bot attacks" when bots come in large quantities to a site. You won't see this in Google Analytics or Adobe Analytics unless you are looking at hourly views; if you are looking at daily charts you may see elevated traffic for the day but you won't notice anything wrong. You will certainly not know what portion is from bots, or what bots they are.

There are also orange bots (ones that declare themselves) and yellow bots (search engine crawlers). Publishers want search crawlers to visit, so their content can be properly indexed for organic search; but usually search crawlers should not amount to more than 1 - 3% of the traffic to a site. In the chart on the right, you see when a declared bot started to come to a site and keep coming back over and over; and the quantities are large. By seeing what bots these are and where they are coming from, the site owner can decide what action to take -- for example they can block or filter them. Blocking means they don't let these bots load the page; filtering means they let the bot load the page, but ads are not called.

As you can see, being a good publisher doesn't just mean they have low to no bots. Hopefully you also realize that it is important to know WHICH bots are visiting, HOW MUCH bot traffic there is, and WHERE the bots are coming from. The fraud verification tech companies are useless in this regard when just reporting a percentage IVT; you can't troubleshoot anything with just a number like that.

A counterintuitive example

What about this example? Do you think it's a good publisher? Would you buy ads from this site? Make a guess first, before reading on. This chart appears to show a lot of light blue (likely OK). So at first glance you might think it is fine to buy from this site. But look more closely at the green bars at the bottom. Those represent hourly volumes. You will notice that it's the same quantity of impressions every hour of every day, with no changes. But remember, humans sleep at night, so there should be natural dips in volume in the overnight hours. You won't see this in Google Analytics (unless you explicitly generate hourly charts ). This is a case where I would recommend not buying ads from this publisher. You can of course do more investigating and even ask the publisher why we are seeing strange traffic patterns -- i.e. whether they are buying traffic. Depending on their answer, you can decide whether to buy from them any more.

The conclusion and so what?

Let me recap. We've seen a simple example where the site had lots of dark blue (humans) and little dark red (bots). We've seen a more common examples where even a good publisher has a sizable portion of bot traffic. They didn't want the bots; they didn't buy the traffic. But bots hit their pages anyway because their site is publicly accessible on the web. So a good publisher is NOT defined by how little bot traffic they have. And finally, even if we see a lot of blue, we don't just assume the publisher is good. We look for other supporting details. In this case, the volume bars gave it away. Something was wrong with the traffic patterns. So we cannot say this is a good publisher even if the charts have a lot of blue.

What are your takeaways after reading the above? Hopefully they include something like the following: getting a single number %IVT is useless because it doesn't tell you what bots they were (some bots are good, others are not). You can't tell if a publisher is a good publisher just by looking at the percentage IVT. A publisher can be good but still have a lot of bots on the site - like scrapers and search crawlers. They didn't buy traffic. The key point is to have the analytics in place so you have enough details to know what is truly going on. Google and Adobe Analytics do not label bots or humans. FouAnalytics does. The fraud verification companies don't label humans. It's pretty clear they don't label bots well either, citing around 1% for the last 5 years. Data has shown they can't even detect "fartbot" correctly and most of their detection tags are blocked or stripped out so they are up to 99% blind. (Be sure to try the "fartbot" experiment yourself. Instructions here https://www.dhirubhai.net/pulse/black-box-fraud-detection-hole-your-money-use-analytics-augustine-fou

FouAnalytics is analytics, not fraud detection. Just like you learned how to use Google Analytics, you learn how to use FouAnalytics. The on-site tags go on sites to help analyze bot traffic coming to the site; the key is to see which bots they were, where they are coming from, and how much there was. The in-ad tags go in programmatic ad impressions so you can see where your ads went and whether bots or humans caused the ads to load. Even if you used to pay for fraud verification before, you have the opportunity to ditch those costs and upgrade your tools.

"When you can see better, you can do better."

P.S. I do not charge publishers to use FouAnalytics.