What's wrong with Search Engine market share data?

The latest market share data for search engines, as updated by Statcounter for April 2024, has just been released. My social media circle, primarily filled with SEO experts, exploded. Statcounter's report indicates a significant drop in Google's desktop market share by 6.5 percentage points. Such a decline has never been recorded in Statcounter's records (running since 2009).

If this were accurate, Google would begin to lose its dominance to Bing and Yahoo. The SEO community is buzzing. "Unprecedented," someone tweeted. Another confirms that this aligns with what they've been observing for a while, and at last, the data is catching up. Another person comments on the stock prices and the possibility of big drops coming. If only it were true... But it's not. Allow me to explain why this data shouldn't be trusted.

Statcounter - the China issue

Recently, I have grown fond of public speaking, and my talks are all on International SEO. I typically cover Google, however I try to drop a different Search Engine name here and there. This means I update the Search Engine market share data on a very regular basis. Over a year ago, the alarm bells started ringing. Supposedly, Baidu, the Chinese Search Engine giant, started losing the home market share. In a span of a few months, Baidu would have lost 30% percentage points, to Yandex - the Russian Search Engine.

Boy, oh boy, wouldn't that be unprecedented to lose this much of market share in a country of over a BILLION netizens? If only it were true...

Note that the chart above is historical, the screenshot has been taken by Jademond Digital in April 2023. So, how does this data present today?

I went back to Statcounter to check the current Mobile Search Engine market share as of April 2024, starting from Jan 2021, to replicate the original statistic. Not only is there only an insignificant drop in market share noted around the April time, there is no Yandex in top 5 search engines AT ALL. The data has clearly been amended. It might be that the intervention by Marcus Pentzek - 潘择科 has been reviewed by Statcounter.

Few points on the matter that require extra explanation:

• I believe that it's not a coincidence that Statcounter picked up on this data trend in China, on Mobile, in March-April 2023, concerning Yandex. Without going into geopolitics, there indeed has been an influx of Russian speaking searchers in China in that time period. Interestingly, the whole of Russia's internet population is 130 million, so 9x less than China's. Even if a big fraction of the Russian netizens have actually crossed the borders to activate their searching, they wouldn't have caused such an impact on the market share data, there is just no way. My theory is, there is some explanation as of why the data was impacted that way exactly at that time, it just doesn't justify the reported scale.
• Geopolitics aside, it is important to mention that China is and has been the hardest country to report on for a very long time. Let's quickly recap why. The Great Firewall of China is an internet censorship and surveillance system implemented by the Chinese government to regulate and monitor internet use within China. It is a mix of technological and legislative measures to block access to foreign websites and services, while promoting a controlled and censored internet environment within China. This means that almost every external data source fails to provide nuanced data on China.

Statcounter - still struggling in China

Even though the initial Baidu vs. Yandex data confusion on Mobile seems to have been manually adjusted or somewhat fixed, Statcounter has still encountered (pun intended) a significant issue when reporting on China. This time, let's look on the Desktop data, over the last 15 months.

When looking at this chart, first thing that comes to mind is - it's not legible. It seems like every single month has a new Search Engine surprise. Let's break it down:

• We start the year 2023 with Baidu being the top Search Engine on Desktop in China, but somehow with only 37% market share. Second in line is Sogou, with 28% market share, and Bing in the third place, with almost 24% search market share.
• Over the next couple of months, Bing goes ahead and takes Baidu over. That's right, the data suggests that the government-backed, technically advanced local Search Engine Baidu loses its dominance to Bing, almost overnight. Suspicious much?
• June 2023 data even suggests that Baidu dropped to the third position that month, with Bing and Sogou standing just a few percentage points above Baidu.
• Suddenly Sogou drops down to 5-8% market share range from August 2023 onwards. This level is actually believed to be truthful and correlates with actual analytics data from the tracked Chinese websites.
• Suddenly we have a 4th player in the game, Haosou. To those of you that observed Chinese SEO for a while, it might be known as 360. It has its ups throughout 2023, but it really comes to shine in March 2024. As much as I love the idea of Baidu dominance being broken down, let's be realistic: no search engine can just climb 17% percentage points, from 13.88% to 31.13% (Jan to March 2024). That's a 124% increase!
• All these elements should sound unbelievable, however there is one more that's going to blow your mind. Google jumps to a whopping 12.79% market share this April 2024. Wait. What? Wasn't Google banned in China, due to the already mentioned Great Chinese Firewall? The answer is yes. Google is not available in China. That's the reason why it has no presence in this market. For those users who try to access it anyway, VPN is the only way. This raises an important question - how would Statcounter attribute the location of a user to be China, if the VPN has been used? Makes no sense. There is simply no way.

I hope that by this point we can all agree that the Statcounter estimations are troubled, to say the least. On top of that, it simply seems like the sample is so low, even the slightest movement grows into a massive percentage jump or drop. The scale of it all is just not realistic.

How does Statcounter source its data?

Statcounter is a web analytics tool. It is an established freemium solution that could be treated as a Google Analytics competitor. Established, but nowhere close the top analytics solutions. As it comes to the statistics side of the business, Statcounter calculates the search engine market share data by analyzing page views that are referred by search engines to websites that have the StatCounter tracking code installed.

As per Statcounter's own website: "we calculate our Global Stats on the basis of more than 5 billion page views per month, by people from all over the world onto our 1.5 million+ member sites." For context, it's believed there's more than 1 billion websites on the internet. This would mean Statcounter accounts for 0.15% of all the websites out there.

This means that not only is Statcounter a very limited source of information, it can be very easily skewed towards one geographical location or a particular demographics. With numbers so low, it's impossible to make it statistically significant.

What the Chinese SE data should teach us about the new Global data revelation

The Chinese Search Engine data chaos is definitely showing us that there is an underlying issue with how the data is collected. It's very likely that no other country in Statcounter's database will be subject to such a faulty trend fluctuation. This being said, I think it is also clear we should treat Statcounter's data with a lot of caution. Clearly, looking at the Chinese data, the Bing prominence is being overvalued in this case. My take on it is that Bing is being overvalued globally in this case as well, as the demographic that uses the websites with the Statcounter Analytics tracking code is a very particular one. And don't get me started on Yahoo.

Is this to say there is no drop in Google market share at all, whatsoever? No, probably Statcounter's data picked up on a very slight trend. I just believe it is not as extreme as presented in their charts, due to the fact their user base is not as representative as we would like it to believe.

One more thing that I want to throw in here. Let's remember we are entering Cookieless era. Chrome turning off cookies for 1% already, many new browsers going the Cookieless altogether. This will affect our data more and more every day. It certainly would have affected Statcounter's data.

Let's treat it all with a pinch of salt.