Afraid of Bing's New Chat AI? Don't Be (Yet).

I was recently granted early access to Microsoft's AI chat enabled search (as a result of opening the Bing app on my phone a few times a night for two days), and was eager to get acquainted with the new Bing experience, as it is currently the buzz of the SEO/marketing space.

I started off with some general questions and was impressed: the answers were correct, the annotated sources in the answers were relevant, and the conversational manner of my interaction with Bing's new feature was refreshing, if a bit terrifying.

Talk of massive disruptions and the nature of search changing are still the buzzwords in the industry, and at first, I wholeheartedly believed it. Then, I started to dig a little deeper.

First off, a quick explainer how Prometheus (the technology that allows Open GPT to be used on Bing and Edge), works. Prometheus uses Bing Orchestrator to take in a user query through AI chat. Orchestrator then consults Bing's Index & Rankings, while also consulting with the Open GPT AI to return a chat response. The technology is pretty incredible, as is the fact that the process only takes a few seconds to generate a response.

However, once I started typing in more precise search queries, longer tail keywords for instance, things started to get funky. Working for CARFAX, my first instinct was to see how Bing's Chat approached the customer journey when looking for a used car to buy.

I got a few sites to choose from and an ad (ew), as the response. Down the rabbit hole we go.

Choosing from the pre-generated prompts above, I chose "A sedan". Again, pretty easy question to answer with some correct annotations. Now I pivoted and indicated that none of these interested me and I wanted to find a used SUV with some specific features instead. (my jump in prompts/answers is due to asking for specific features on a sedan and then seeing how the AI would react to a change in vehicle type).

Again, I was in awe that the AI was sophisticated enough to consistently list out some cars with their inventory info right in the chat. Then I started looking into the annotations, and that's where things get interesting...these cars don't exist.

You can see with the above listings that edmunds is being cited as the source for those listings, with the bottom "foot note" webpages being used as the source of information as well. Except that edmunds article is a general "Best SUVs" listicle, with no price, mileage, or hell, even listings on the page.

The AI is taking what's Indexed in Bing, considering the information in those indexed results, and creating an amalgam of what it thinks best answers the user's query. After searching all the annotated pages, the AI was taking price, mileage and year information and applying it to the "best used SUVs" article vehicles, in the process creating car listings for cars that don't exist, just what it thought would be most appealing to me.

And the links at the bottom and the clickable ones in the answer text? They took me to general pages on CARFAX or Carmax that contained the mileage and pricing information (in some instances); some of the mileage and pricing info I couldn't find, even after manually reading and scraping the pages.

Now, I understand that this AI is still in its infancy and is always learning, and I am not denouncing Prometheus and the technology behind this, it is amazing. However, it is still very young, and in that respect, prone to errors.

The "grounding" process that Microsoft is using for this AI chat feature (the process of using both the index and AI to "ground" the results so that they are more reliable), is not working, at least in my super specific example.

I know this is only one example (it has happened with multiple car related purchasing searches in my experience), but it's alarming that someone may take those listings, or whatever other information the AI annotates and spits out as fact, much like people take featured snippets and other information on the web at face value.

So with that said, at least where I'm operating in the SEO world, Bing's Chat is not a game changer for those looking to purchase a car. It's more of a game changer for those who run recipe sites, or strictly informational/educational sites where there pages rank highly in Bing, can be easily scraped by the AI, and presented with some conversational fluff around them.

That's all the AI is at this moment; conversational window dressing around the top results in Bing's index (or an inaccurate amalgam of many of them).

Will this be the case in 5 or 10 years? I have no idea. But or right now, there's no reason to get stressed out.