Design patterns Journey: Caching Layer in Replicated Services

Should you take cash out of ATM each time?

You would like to go to a market and purchase couple of goods from different shops.

Which scenario you like?

You go to SHOP1 and then get the bill and then go to nearest ATM and fetch money and come back and pay

You go to SHOP2 and then get the bill and then go to nearest ATM and fetch money and come back and pay

You go to SHOPn and then get the bill and then go to nearest ATM and fetch money and come back and pay

or

You go to ATM and fetch an anticipated amount of cash and have it in your Wallet

You go to SHOP1 and then get the bill and pay from your Wallet

You go to SHOP2 and then get the bill and pay from your Wallet

You go to SHOPn and then get the bill and pay from your Wallet

What you have observed here , going to ATM fetching money each time has so many disadvantages. In most of the cases having the money fetched and available in your pocket/Wallet works very well. It also increases the speed of the entire purchase.

Caching Layer is your Data Wallet in cloud

You have already learned one important design pattern. Caching !!!!.

In cloud -

Money is data

Wallet is the cache

and you are a MICROSERVICE or MICROSERVICE chain :)

ATM could be a DATABASE or an EXTERNAL SERVICE.

The only difference is - data is as good as its validity or relevance .. and it can go stale

Stale data is comparable to demonetized money - only the point to be noted that demonetization frequency might be very less compared to data going stale probability.

In cloud world, a caching layer can make a great deal of sense. A cache exists between your stateless application and the end-user request. In some frequent calls an additional call to DB might just bring down user experience so much that performance tests may suggest to bring in caching.

In memory Cache

If its a replicated microservice and the cached data can differ in each replica but still relevant - then in memory cache would work just fine.

For example : you could cache all the authentication TTL tokens generated by a particular replica in memory.

Distributed Cache or Cloud Cache as services

In some cases you can not simply cache different values in different replicas.

In such cases you may go for distributed cache mostly for cloud cache servuces like https://cloud.ibm.com/docs/databases-for-redis?topic=databases-for-redis-redis-cache

Comparison of In memory cache and cloud based cache services

• In memory cache gives best response times
• but you need to maintain the in memory cache code where as cloud cache service is like any other cloud service and cloud provides manages it. Especially you can not afford to have security issues with an in memory cache that stores auth details for eg:
• Stale data management : Its not easy to say which strategy makes it easier. But if you need in memory cache of all replicas to have no stale data in comparison with other replicas - you might need to build some extra mechanisms such as sending messages between replicas in some cases via kafa for example. In some cases you can go for WRITE THROUGH CACHE approach in which microservice will alwats write everything to cache and DB but reads from cache alone. This should be only used in case stale cache data is major issue as well response time

• In cases of cloud cache service - sometimes you will have to have special mechanisms to clean the cache. But in memory cache can be simply cleaned by a restart of the replica
• If you want data to be available in cache after restart - you should probably go for cache service. You could optionally load in memory cache on start up.
• TTLs -> You should use this wisely in both cases. A cache miss means lower performance. A stale data means wrong data served to user. Set TTLs accordingly. TTLs must be lower in cases of security related data (auth tokes for eg). TTL also must be related to memory allocated to the replica and growth pattern of memory of replica
• CPU utilization : Usually cache services solves the issue.