Google’s AI Overviews: Early Studies, Tools and Testing

Since Google launched AI overviews, we have kept tracking how they evolve and develop.

There has been a lot of criticism of AI-generated answers people see in Google searches because they provided false and sometimes life-threatening answers. @Goog_Enough was set up to curate these complaints.?

Google listened to users’ feedback and fixed a lot of issues, mainly limiting the visibility of AI overviews in search.

Here are a few earlier studies and tests around public AI overview feature rollout:

1. 15% of Queries (or Less)

Earlier studies on the visibility of AI overviews found that it currently shows for 15% of search queries.

I’ve seen more studies showing that the frequency of AI overviews is getting even lower.

Note that AI overviews currently show up for searchers who are located in the US and logged in to Google. So in reality, the AI overview visibility is lower (non-logged-in people don’t see them).

2. Public AI Overviews vs Labs

Searchers who opted in for SGE labs earlier may see different results from what other people see. To see public results, you need to either opt out of labs. Another option is to add an alternative Google profile in Chrome to test both.

Lab participants see the “Search labs” note when searching.

People who are not participating in the lab experiment may see different results. For example, public AI overviews may include references before the “Show more” link:

If you were never part of the SGE lab experiment, you are seeing public results, so no action is required provided you are signed in to Google.

3. Tracking AI Overviews

Not many SEO solutions offer AI Overview tracking solutions. One I found and was pretty happy with is called ziptie.dev?

Import your list of important queries and the tool pool:

• Where you rank
• If you are featured
• If there’s an AI overview showing up
• If you are referenced from the AI overview

Click any query on the list and you will see which URLs are referenced from the AI Overview and whether they rank organically:

4. Getting Linked from AI Overviews?

We’ve long known that getting your site listed is an AI overview is pretty doable. From the SGE patent, we’ve known that:

• AI answers summarize existing Google’s search results
• Links in the overviews are generated after the summaries are created.

This seems to be still true because a quick test by Cyrus Shepard showed how easy it is to include your link into AI overviews. If your page is ranking for a query triggering an AI overview, update your page to closely match the text in the AI overviews, and you may quickly get included in it.

There are two caveats to this test:

• The tested page was no longer featured
• The AI overview inclusion was temporal and quickly lost.

The page disappeared from the AI overview too fast for us to have any evidence of how effective it is to be included in the new feature. The quick dynamics of the test show that it is probably too early to experiment with your important queries. Let’s keep an eye out and see!