Slow REDIS config cache in Magento 2 - source and possible solution.

The problem:

In one of our projects, MultiStore, I've found out an interesting thing regarding how Magento 2 works with REDIS cache for config storage.

Situation:

• two production servers
• REDIS on a separate instance (for sessions and config cache)

During the day, this is an average REDIS hGet load time on every single page (even in customer/section/load operation). This method is called 217 times (on average)

In the last few days, I've tried to resolve this after hours and I had to dig into the entire process of how the cache instances are created, managed, loaded, and saved by the platform.

What I've learned is the fact that Magento, during System configuration load, done by this class \Magento\Config\App\Config\Type\System Magento loads creates and loads all configuration scopes: default, websites, and store views. What is interesting is that the same cache is created on both IDs and Website / Store Codes.

It results in creating a lot of cache entries for System configuration:

• Default
• 2x for All websites ( ID and code)
• 2x for All store views ( ID and code)

In the end, it looks like this ( REDIS keys)

In our scenario, we have approximately 30 store views. As a result, Magento fetches 121 cache entries during its system initialization process. If we assume each request takes 1 ms, it results in the page taking an additional 121 ms to load.

Yet, this isn’t the full picture of what's retrieved from the REDIS cache. During a single-page rendering, Magento also attempts to load more cache entries. For this particular project, another 67 cache entries are loaded.

Ultimately, the system fetches around 200 cache entries from the REDIS config cache, which adds 200ms to the load time of every uncached page. Notably, some of these entries are loaded multiple times.

I've tried the following approach (DEV site)

• replace REDIS with https://github.com/dragonflydb/dragonfly - it is 100% compatible and is in fact faster (a little) but it has not brought any improvement in this project so far.
• used L2 cache for REDIS to store values locally in the RAM drive.

However, it didn't solve the issue and I was only getting a difference between 15-20ms for a single page load - no great IMHO.

The best solution, so far, to reduce this load speed was simply to remove the REDIS config cache from env.php. At least for this project.

Possible solutions:

I've come up with a few ideas I'll try to experiment with in my free time (so probably never ;-)):

• removing duplicated cache entries for websites and stores and using only IDs for cache keys. This will reduce the page load on multi-store environments a LOT. Maybe something for Mage-OS ?
• Creating some sort of a cache registry on the PHP level so we don't load the same cache entries more than once.
• gathering all data from REDIS / config and creating a configuration PHP file that is loaded with the application and it will store all config cached in one giant array. Yes, the file will be 30-50MB (at least) in size so I'm not 100% sure how this will affect the PHP processing and application speed.
• using APCU instead of REDIS to store/retrieve config cache entries

I wonder how others approach this issue, Share your ideas, and let's implement a solution that will benefit everyone :-)