Why are some of the pages on my website not indexed by Google?
Nikki Pilkington
Non-wanky SEO and SEO Content. They say you can't know Technical SEO AND copywriting - I can. And do.
A frequent question over the years, and in the past the only answers given would be an educated guess at best.
But now, with Google Search Console, not only can you find out which of your pages aren’t indexed, you can find out why.
Navigating to the Indexing –> Pages section of GSV, you’ll find a chart showing how many of the pages on your site are indexed by Google, and how many aren’t:
(Hopefully yours won’t be quite as scary as this – remember, the site we’re using as an example in this blog post has had no SEO work done at all.)
From the above graphic, we know that 535 pages aren’t indexed, for 7 reasons. Luckily, Google tells us those reasons too!
So, let’s go through the reasons that all those pages aren’t listed.
Excluded by ‘noindex’ tag
This means that either you or someone in charge of your site has specifically told Google not to index these pages. If you have a WordPress or similar blog, this will usually include category and tag pages, feeds, and pages that don’t need to be indexed (such as your Terms & Conditions and Privacy Policy pages). It can, however, include pages you really do want to be indexed.
What to do:
Check through the pages listed to make sure that nothing is being restricted that shouldn’t be. If you see pages that shouldn’t be restricted, talk to your web developer or an SEO expert.
Page with redirect
This often happens when you have changed URLs on a website. For example, most of the issues with the GPT Genies site are because we restructured the URLs of the blog posts, and Google has decided that the new URLs are the correct ones, so the old URLs it still knows about are redirects. To avoid indexing both, Google is ignoring the old URL and deciding that the newer one is the right one.
This can also happen when you redesign a website and use redirects to point old pages to new ones.
What to do:
Make sure that all of these URLS redirect to the correct pages through?proper permanent 301 redirects ?and delete the old pages.
Alternate page with proper canonical tag
Google has recognised this page as an alternate version of another page. It could be the mobile version of a page already indexed, or vice versa. Either way, you can ignore this section as all of these pages have alternate versions that should be indexed.
What to do:
There is nothing that you need to do here.
领英推荐
Not found (404)
When Google tried to index these pages (usually from a link elsewhere) the page was not there. It may have been renamed, deleted, or the original link may be incorrect. If the page listed has a newer version, the best thing to do is set up a 301 redirect (find out more info on 301 redirects here), but if it has been deleted or the original link is incorrect, the best thing to do is wait. Google will check occasionally to see if the page has a new redirect or is still showing a 404 error, and eventually just stop checking altogether.
What to do:
If the page has moved, use a 301 redirect to the new location. If not, eventually Google will stop checking the page.
Soft 404
A soft 404 is a page that tells Google the page has been successful, but is actually an error page. Often (but not always) custom 404 pages – where you tell the user that the page isn’t found but give them some links or a search box to help them find it – give this response. My advice is talk to your web designer on this one.
What to do:
Either read this?Google document on soft 404s , or send it to your web design expert.
Discovered – currently not indexed
These are pages that Google knows are on your site. They just haven’t crawled them properly, either because crawling them would have overloaded your website’s resources or because your crawl allocation has been reached.
What to do:
You could just leave this and wait for Google to reschedule the crawl, but if it’s a page you want to be included in the Google index, then you can submit it manually.
Crawled – currently not indexed
Google has found these pages and crawled them, but for one reason or another has decided not to include them in the index. This could be for a number of reasons – poor copy, poor?E-E-A-T ?signals, duplicate content, and more. Take a look at these pages and see what you can do to make them better before resubmitting to Google.
What to do:
Google advises doing nothing, as the page may or may not be indexed in the future. I would recommend checking the content, updating and rewriting so that it is fresh, and then submitting manually.
There are a few other reasons that pages may not be indexed, but these tend to be the main ones. If your Google Search Console is showing different errors, you can usually find out the reason why by clicking on the error, then clicking ‘READ MORE’.
You can read more about Google Search Console here , or feel free to DM me if you need help with GSC or any other SEO issue.
Marketing expert for built environment firms who want results | Marketing strategist | Speaker | Trainer | Mentor | Non-Exec | NED
1 年Wow this is super useful, I was just looking at this yesterday on a client's website
Website Conversion Optimisation Specialist | Nerdy nomad
1 年?? Nikki Pilkington - thank you so much for this article! I've been trying to find a concise way to explain this to some of my clients for ages - they often panic when they see an indexing 'issue' that actually they don't need to be worrying about. I'll be sending them this article in the future!