The importance of caching for SEO

SEO can be hard for outsiders to understand. In larger enterprises, organic search professionals often struggle to convince other departments of the importance of certain aspects of their work in achieving positive results.

As an SEO professional of over 10 years, one thing I have repeatedly seen is a failure to appreciate the importance of just how technical SEO really is.

In the view of many people looking from the outside, SEO is just another marketing channel that can be switched on and off, in the same way as PPC, programmatic or social. But this could not be further from the truth.

Although it is true that SEO is a highly-effective acquisition channel, optimal performance in this channel is highly technical and rooted to the very code of the website, as well as the server settings, and even Google’s own technical view of your website.

Nowhere is the technical aspect of SEO truer than when it comes to caching. Get it right and your site can compete on Page 1 for multiple highly competitive keywords. One false move, and that same site can struggle to break the top 100 – even for your own branded keywords.

It doesn’t matter how wonderful your content or backlink profile is. If Google struggles with your cache settings, it will not rank you.

In this short article I would like to explain a bit more about caching, how exactly it helps, and how it can hurt your rankings when implemented incorrectly.

Let’s start with the basics: what is the cache and what type of caches we need to recognise.

What is caching?

Caching is a way of taking certain website resources such as images, video, CSS and HTML and storing them in a temporary, easily retrievable format. In practical terms, it means that your website can be delivered much faster to a user.

In 2020, we all know that speed is an important ranking signal, since Google doesn’t want to recommend a slow site to users. But on top of that, if your cache is off, Google itself will dedicate progressively less crawling budget to your site. Each time Google crawlers come to visit, they will be able to visit less pages of your site, find less of your great content, and less of your inner links.

Sub-optimal caching isn’t just a bad user experience, it is a bad Google crawler experience.

Why is this the case? We need to consider the 3 types of caching out there, and how they influence each other.

Server cache

Optimal server caching is paramount for a successful user experience (and as we shall see, ultimately, successful SEO results). The good news is you can control these settings yourself

You set different cache times to different files (and file types) on your website based on the user’s need to see the latest version of that file. Different file types should be cached differently. For example, your front-page content (the HTML) should probably be cached more often than the background image, or stylesheet.

How often should that be? It depends what your site does. If it’s a news site, you might even want to refresh it every time a user visits. It depends on your niche, the content of your site, and even what your competitors are doing.

To check your website’s or your competitor’s caching times, I recommend you visit this website: https://www.giftofspeed.com/cache-checker/

Browser cache

Browser caching occurs on the individual visitor level and is not fully controlled by you – although you can influence it by your server caching settings, which tells the browser whether or not it should fetch the latest version of each resource.

Browsers such as Google Chrome and Firefox typically cache so called static assets (images, CSS) which do not change too often from visit to visit, and so can be loaded much faster, as they don’t have to be downloaded from the internet again.

Although you can’t control how a user sets the browser cache (they could do a hard refresh each time if they wanted to), in the real world, browser caching is still very important, as this is how your browser remembers and stores the websites you have visited in their local memory and helps the website to load faster.

Google cache

The Google Cache is a snapshot of your page’s content and is taken when Google last crawled your website’s page.

Although Google doesn’t disclose exactly how their caching works, experiments suggest that the Google crawler stores its visits to your site in a kind of ‘virtual’ browser cache. Each time it crawls your site, it seems to use what is already in its virtual browser cache as a baseline, allowing it to progressively crawl more each time.

IMPORTANT: By the same token, if your server caching settings tell users to fetch the latest version of every resource on every visit, the Google crawler wont be able to build on the previous crawls – meaning less of your website will be crawled. If your server settings tell users (and the crawler is a virtual user) to refresh every visit – you can expect to see your rankings drop.

This cached version is also what Google uses when finding results to show in the SERPs for certain search queries.

How to check when and how Google last cached your website:

Use the command cache:website.com/page and you will be able to find out the last time  your (or any other) website or page was cached.

Possible cache errors:

When rankings across your site mysteriously and continuously drop, and there is no sign of a penalty, caching might be to blame. Here’s what to look out for:

1.)   404 cache error

2.)   Caching the wrong page so instead of caching xxx page yyy is cached in results, which means the contents of xxx are not being crawled and cannot rank

3.)   Wrong cache times (check how your top competitors do it and make sure you are not too much of an outlier)

What to do if you find any issues with Google cache

1.     Check the canonical tag and make sure it is used correctly

2.     Check your sitemap, and if necessary, generate a new one

3.     Check Google search console for any alerts or messages

4.     Check the Google search URL inspection function

5.     Run technical SEO diagnostics using tools such as Screaming Frog

6.     Check you server cache settings

If you do not want your website to be cached or appear in archive.org

There are cases when you do not want your website/pages to be cached, like dynamic pages with constantly changing contents.

In that case use the following tags:

content=”noarchive”

content=”no-cache”

In summary, keep in mind that cache is vital for your SEO health and without a properly set up cache on server level, your website cannot perform.

So if you see a drop in rankings, make sure to run cache tests as a part of your SEO audit.

Speaking from experience, keeping on top of your caching can save a lot of pain and detective work.

Happy SEO’ing