Cadence #24 - Index? Now. CrawlNow.

This week's cadence is brought to you by Mirador.

Mirador is an all-in-one management for your Google Business Profile listings. Manage your listings from a single dashboard whether you have 10s, 100s, or 1000s of listings.

Currently in Alpha, you can get started from just $35 per month.

Happy Sunday everyone!

Welcome to another cadence, and this week we're looking at IndexNow.

I want to start with a caveat. I do not think IndexNow is bad, and I agree it is the future of how search engines will scalably, efficiently, and effectively discover and crawl content on the internet creating a new relationship paradigm with webmasters.

I just disagree with the name.

IndexNow uses an automated web crawler to discover and explore your website, analyzing page content, metadata, internal linking structure, and more.

It then submits your site to an expedited crawl queue at major search engines, notifying them of your new content so they can crawl and index it sooner.

Search engines likely prioritize sites for crawling based on many factors, and there is little evidence that IndexNow has a meaningful impact.

Prioritizing Crawl Queues is Not Indexing

IndexNow does not actually index websites itself. It simply adds customer URLs to the priority crawl queue of major search engines like Google, Bing, and Yahoo.

While this may allow sites to be crawled and indexed faster, it does not directly index the sites.

The search engines still have to crawl, analyze, and categorize the content to add it to their indexes.

Limited Resources for Priority Crawls

Search engines receive billions of URLs to crawl and have limited computing resources to dedicate to their crawlers.

They cannot realistically prioritize every URL submitted to them through services like IndexNow.

At best, IndexNow URLs may receive a minor boost in the crawl queue, but they are still competing with countless other URLs for crawl budget and resources.

No Guarantee of Indexing or Ranking

Even if IndexNow is able to elevate a site in the crawl queue, there is no guarantee the search engine will choose to index the content or rank it well in results.

The content and quality of the site ultimately determine if search engines find it useful and relevant to index and rank for certain queries. Indexing and ranking are complex processes that involve many signals beyond simply crawling a URL.

If you're having issues in getting your web pages indexed, or you've seen decreases in your indexed pages/traffic and need a hand, please feel free to drop me a DM.