SearchGPT vs. Google vs. Bing: Search Results Review
The role of AI in SEO has kept rapidly evolving over the past few months.
Google formally launched AI Overviews in May, and Bing launched its generative search pilot in July, with a full launch at the start of October. On July 25, OpenAI announced a SearchGPT prototype . Then, on October 31, it launched ChatGPT Search .
OpenAI are calling the new product, which I’ve tested for this article, ChatGPT Search, but parts of the industry are already starting to use the names ChatGPT Search and SearchGPT interchangeably, as they are effectively one and the same.
Both the Google and Bing AI launches, alongside the increase of multi-modal search (e.g., TikTok), have been impactful, but the ChatGPT Search launch feels different.
ChatGPT already drives traffic to websites, and as of August 2024, it reported 200 million weekly average users (WAU) – this is all while AI is gaining momentum and user trust, with skepticism still a barrier to mass adoption.
In February 2024, Gartner predicted that “search engine volume” will drop 25% by 2026 .
Having played with SearchGPT, this prediction feels like it could become a reality.
ChatGPT Search Observation Summary
Keep reading below for the full article and query breakdown. You can also watch Dan talk about the launch of ChatGPT Search in his interview with Shelley Walsh:
Increasing Bing Market Relevance & Importance To ChatGPT Search Inclusion
Bing is fast becoming a search engine we need to change our views on. For as long as I can remember, Bing has always been an afterthought to the industry and has only really been prominent in specific sectors and target markets.
While its direct market share might not be growing to rival Google in that sense, the fact that Bing is being utilized by multiple large language models (LLMs), and ChatGPT Search is no exception.
OpenAI has been using Bing for a long time, so its inclusion in SearchGPT isn’t a revelation, but a reinforcement of Bing’s evolving place in the overall “search” landscape. We need to stop just comparing search engines and their direct market share.
This means that websites (and webpages) not indexed by Bing will not appear in ChatGPT Search.
Anecdotally, traffic from LLMs to all websites I have GA4 access to has increased exponentially over the past three months.
Of all LLM traffic, ChatGPT accounts for 31% of my website (year-to-date), with it as much as 60% on some websites alongside Perplexity, Claude, Copilot, etc.
However, this doesn’t necessarily mean you need to rank high in Bing to be included. Much like Google’s AI Overviews, ChatGPT selects sources outside of the top-ranking results.
It’s also worth noting that while Bing is important to the SearchGPT ecosystem, it isn’t the only source.
Comparing SearchGPT, Google, And Bing Search Results
To see how the search results in ChatGPT compare to both Bing and Google, I’ve looked at a number of queries across Local, Your Money or Your Life (YMYL), ecommerce, and some informational/time-sensitive searches that I’ve personally performed over the weekend.
I’ve tried to live with ChatGPT search for 48 hours rather than subjecting it to a barrage of random queries.
I’ve not recorded all of the queries in this article, but I have included the key ones as well as some oddities and differences.
The number of webpages referenced that don’t appear in the top 100 classic ranks of Bing really highlights that other sources are at play here, and that optimizing for Bing doesn’t mean optimizing for ChatGPT and other LLMs directly.
In AI Overviews, when a webpage is referenced outside of the top 100 classic results, I feel we just assume it’s come from Google’s overall database. After all, Google crawling and building a database(s) of pages on the internet is known – but could OpenAI be doing the same?
ChatGPT Search “Maps” & Local Search
Local queries are important to a number of businesses. SearchGPT (at the time of writing) has a somewhat limited local search experience.
When searching for [enterprise technical seo agency], it provided me with three results: Two close to my IP address and the third in a random location in the South of England.
There are a handful of “agency hotbeds” in the UK, and a lot of agencies in my local area. So, if SearchGPT were going off IP, it would have included more agencies locally. To then stretch the net further and ignore London in the results for broader results doesn’t make sense.
While none of the three recommendations were wrong, SearchGPT seemed to hallucinate on the second result.
In the first and third results, it added the label “Advertising Agency,” but the second result was given the label “Telecommunication Service.”
I have spent time looking through the agency’s website, social profiles, backlink profiles. I’ve crawled the website with various custom path extractions setup. I’ve looked at all the schema markups that were implemented. I cannot fathom or find a reason as to why it has been classified as a telecommunications service.
When repeating the search for [digital marketing agency in new york], it provides the map, but with eight results all from the same source.
This is less useful as a result, as being a reference for this query is basically a game of appearing in a certain website list – which is a paid membership only.
The website used to compile all the map results ranks No. 4 in Bing classically, so it is at least a high-ranking result – but this just feels like scraping a list on a website and is a lazy result.
Triggering The Map
The map result (as shown in the screenshot above) doesn’t always trigger.
For a number of queries when I specified a location, it just provided the list results without the option to view a map. This is different from the “list” toggle on the Map, as this keeps the map but creates a list underneath it (which is very similar to the Google Map Pack).
Speaking of Google Maps, if you ask SearchGPT explicitly to show you the results on a Map, it directs you to Google Maps:
YMYL Searches
YMYL queries refer to search queries that could potentially impact a person’s health, safety, or financial stability.
It is widely understood (and communicated) that Google and Bing place greater emphasis on these queries and ensure that accurate information is provided from trustworthy sources.
“Remineralizing Gum”
According to Glimpse , “remineralizing gums” saw exponential search interest in October 2024.
As this is an emerging query in the health space, one could assume that the trend will be picked up by websites selling the product (or adjacent products) and producing content around the topic.
SearchGPT feels like it played it safe with this query, only utilizing WebMD and Wikipedia as citations before providing references to a variety of other websites.
“How To Lower Cholesterol”
The response for this query doesn’t highlight anything dangerous or out of the ordinary and is measured in response.
I also recognize a number of the websites in the citations, which help with trusting the information.
Interestingly, SearchGPT is pulling from two separate URLs on the Mayo Clinic website, despite one of them not ranking in the top 100 for Bing.
“Who Can Sign A Contract On Behalf Of A Company”
On the first run of the query, the response contained a more textbook answer, citing the four main criteria under English Law for a contract to be valid (Offer, Acceptance, Consideration, Intent), as well as two additional bullet points around capacity and that the contract itself is legal.
领英推荐
From this response, the cited source URLs and top referenced URLs barely ranked in the top 100 results of Bing:
However, when I perform the same query on a different computer – on the same account and same internet connection – I get a completely different response:
The citations and search results provided also differ, with only one of the original citation sources remaining.
On the second run of this query, even fewer citations ranked in the top 100 Bing results.
Interestingly, when looking for the domains on the second data pull for this query in Google, the domains ranked higher with different URLs than those cited by ChatGPT Search.
“How Many Credit Cards Should I Have?”
Testing financial queries, I saw the closest alignment to both Bing and Google’s first page of results.
Four of the five cited sources in the table above appear on Google’s first page (in the UK) for the query, with the only exception being the bank.marksandspencer.com URL, which appears in the middle of page two on Google.
Ecommerce Searches
Transactional searches in ecommerce refer to queries where the user intends to make a purchase or complete a transaction.
These are often high-intent searches, where the user is actively seeking a product or service, and they’re close to making a buying decision.
In my models, I’ve talked about how AI can’t necessarily satisfy the full extent of an ecommerce query (as AI can’t facilitate the shopping and purchase experience); I’ve classed this user group as “Purchasers.”
The simpler and more open the query, the more likely AI can steer and influence, but as a user narrows down to make a purchase (moves from being a shopper to a purchaser), it still needs to engage with brands and ecommerce websites.
“Best Christmas Gifts For Him 2025”
Another positive point for SearchGPT is that all of the webpages referenced and cited for the query have been published (or at least claim to have been published) in the past month.
An adjacent test I’m running on AI Overviews at the moment for similar queries has content as far back as 2017 being referenced and cited in AI Overviews.
“Black Friday Deals”
Another positive point for SearchGPT is that all of the webpages referenced and cited for the query have been published (or at least claim to have been published) in the past month.
An adjacent test I’m running on AI Overviews at the moment for similar queries has content as far back as 2017 being referenced and cited in AI Overviews.
“Black Friday Deals”
While being very single-source heavy, the second citation for [Tips for maximizing Black Friday Savings] came from an Australian website.
Google’s AI Overviews have received some criticism on X (Twitter) and other platforms from a number of SEO professionals as to the inclusion of Australian sources in non-AU markets.
This could be an indication that SearchGPT may exhibit the same behavior across queries.
Do I Think This Will Disrupt Google?
Google has been facing disruption since the mass market’s relationship with mobile devices changed.
Moving away from desktop and utilizing apps and browsers to research, shop, and be entertained opened the doors for new platforms to compete for screen time and attention.
On a recent podcast I ran, John Mueller said that when Google first started to push towards mobile-first, a lot of SEO pros and webmasters were skeptical and vocal that users wouldn’t shift from mobile to desktop to view their websites.
SEO as a practice is only 30 (or so) years old – and in 30 years, how users interact with businesses has changed drastically.
AI platforms, such as SearchGPT, will disrupt Google as adoption increases and the “early majority” of consumers start to use LLM platforms daily for tasks they otherwise would have engaged the wider Google ecosystem with.
We can draw some parallels to the Model T, and when Henry Ford introduced the car, it was seen as a novelty and not for the masses – with people seeing them as “faster horses” than a revolutionary mode of transport.
As SEO professionals, we can see the “revolution” part of it in motion, but we must remember that to the mass market, a lot of AI is merely gimmicks such as removing things and people from the background of images on phones or using things like Circle Search .
SearchGPT will contribute to the overall disruption, and it might pioneer the “what’s possible, “much like Jeeves was a pioneer in mass market internet search engine adoption.
Also, I don’t feel this would be a complete answer if I didn’t ask SearchGPT directly if it will disrupt Google:
Which Search Is Best?
In testing SearchGPT felt as though it had a better understanding around time sensitive queries and used more recent sources.
Removing visual experience from the equation, SearchGPT is better as objectively you want recent content when you’re searching for information around holidays, gifts, products for “now,” and not viewpoints from over a year ago.
For local search, ChatGPT Search still has a long way to go and falls short of were Google and Bing are.
In my opinion, information was better in ChatGPT Search than in Google and Bing, but not on all queries. Overall I still feel it is behind Google’s AI Overviews and Bing’s Generative Search in terms of answer queries behind text-driven informational results – and the ChatGPT Search interface is a little bit too detached from what we’re used to seeing from a “Search engine”.
Closing Thoughts
Brett Tabke has referred to SearchGPT as the “final” search engine, and I agree with this.
While SearchGPT currently offers certain advantages over Google and traditional search models, it is still a work in progress.
However, its potential trajectory is one of the key insights to take away from SearchGPT.
This development indicates that user interaction with the internet is evolving once again, and as SEO pros, we must adapt as an industry.
The rise of platforms like TikTok highlights the shift toward multi-modal search, urging us to move beyond a narrow focus on the “engine” aspect of our work.
The core principles of SEO, particularly technical SEO, will always remain relevant, as AI crawlers must crawl, identify, and access content similarly to traditional search engine crawlers.
I believe it’s time for us to adjust our SEO terminology, shifting the conversation from ranking to referencing, and realigning what the overall success of an SEO campaign is.
And we finally start taking credit for upper funnel visibility and touchpoints and are not measured solely on bottom-of-the-funnel conversions.
Rather than asking the questions “How do we optimize for SearchGPT?” or “How do we optimize for LLMs?,” we need to be looking at our market and the different LLM platforms and asking questions like:
Source: Search Engine Journal