Is More Caching = Efficient Application?

?? Hi, this is?Navjot?with a new newsletter. I write about software engineering, best practices, industry standards, and career growth. Thank you for being a reader, and making it a family of over 750 frequent readers with over 200k views.?? ??        

For this newsletter, I emphasized upon Caching and how its overdose and inefficient integration can potentially slow down a service instead of making it faster.

Caching is heavily used in the world of Web Applications and Cloud Services.

• It makes our application faster by reducing latencies for frequent queries
• Protects the database from multiple read requests. The cache acts as a protective agent for the system.

So this must mean that the more cache the merrier? Well Maybe...

Caching

Caching in simple terms: To keep a local copy of frequently accessed data close to the processing unit or in a distributed system such that the response can be served directly without recomputation.

Caching helps reduce the load on backend applications by saving recomputation on already evaluated data.

Why do Developers love Caching?

In an article by DEV Community, they implicitly mentioned

"We’ve always put a lot of effort into performance at DEV. We want our users to be able to see their content almost instantaneously when interacting with our content. To do so we’ve placed a big emphasis on caching."

This statement sums up the expectations of all consumers with their applications.

Problem with using Cache?

Cache is not your in-memory DataBase

Once you begin to use Cache for one component you unknowingly bag on the Cache service for storing all relevant information and boom you have created a dependency on a Cache System so heavy that you cannot deprecate and Migragte it without development efforts.

At this stage, your Cache has become an in-memory DataBase which was never intended to be and can harm data quality.

You kept this data in memory because it is critical to applications and needs to be delivered quickly and/or frequently. Otherwise, it wouldn’t be worth the expense.

However inefficient caching introduces potential issues with the consistency of this critical data. An inefficient cache introduces the potential for a mismatch between the data held in memory in the cache and what is persisted in the database and breaks the application's consistency.

Caching is expensive

Caching makes our computation faster, but everything has a tradeoff.

Cache memory is compact and fast and therefore is expensive to procure. Although all new processors have already embedded Caching Memory it only allows you to store data in a few KBs and for smaller infrastructure it is sometimes limited to certain < 1 KB. So if you are trying to use cache for all read queries the Memory is bound to fill up fast which might cause a broken customer experience.

Caching service breakdown?

So you have already offloaded your system’s responsibilities to a cache.

Not to forget you are now adding more moving pieces to your infrastructure and with enough load Cache services tend to fail with CPU and memory outages which is nothing but a heap memory crash.

This will cause cache misses for all incoming requests as all requests first go to the Cache service and then to the backend applications.

Having high cache miss is a horrible customer experience

“Cache misses will add latency that otherwise would not have been incurred in a system without a cache”.

So what you were trying to achieve by adding cache is now gone.

Is Multi-Level Cache the right way forward?

If a single-layer cache goes OOM why not use Multi-Level Cache? Multi-level cache is designed to bridge the gap of low capacity of Cache Memory.

You would argue that we can keep a multi-level cache in the system and have a good night's sleep?

?Well, microservices are human-made, and just like humans Services fail too. You have already added multiple levels of dependencies by adding multi-level cache and now you have to

• monitor each one of them to ensure they run fine.
• It not only adds to operational cost but adds significantly to the infra cost as well.
• If One layer fails Good luck setting up the system again.

Advice from seasoned engineers.

In discussions with some senior engineers of mine.

I have always been advised to "Use Cache as the last resort."

It always looked like an anti-pattern to me but the arguments I got from them put had put things into perspective.

• Cache should be used only when you see no scope for improvement in your system.
• ?If you see your DB read has a high latency look at the query and DB tables and try to optimize them first.?The trick is often to break down each query. Do you need all the columns? Could you add an index on the most important column?
• If you see your computation taking time evaluate the choice of the underlying infrastructure being used with the algorithm of choice.

The gist was simple.

Caching in some places can be a duct that fixes your broken pipe quickly but can cause more spillover if overlooked.

TLDR

?Caches are probably the greatest thing since sliced bread. Use it right and they help perform, scale, and be available better. Abuse it and you're a toast.

PS: I hope the article had some valuable insights. If you have any queries feel free to comment down below. Till next time ??.