The need for speed: improvement for Magento / Adobe Commerce / MageOS to speed things up!

It's chilly, it's drizzling... so what's a coding ninja / wizard to do after a soggy stroll with a canine friend? Why, dive into a debugging frenzy, of course! Let's crank up the speed on everything because, honestly, what else is there in life but to make code run like it’s late for a date with destiny?

Configuration cache processing

A few months ago I've written this article about Magento and Redis https://www.dhirubhai.net/pulse/slow-redis-config-cache-magento-2-source-possible-solution-winkler/.

Long story short: Magento loads the same configuration multiple times and for large stores with many store views it can be a performance killer. Does not matter if you go with REDIS or file cache.

Today I've decided to take another look at issues (challange / problem), on sample data with a single store instance and see what's going on there. After some digging and profiling with the Blackfire.io tools I've decided it is time to optimize this method:

So what this method actually do? It is very simple:

• it reads the configuration from cache (or files) that is stored as a multidimensional array (like env.php / config.php files)

• and converts it into one-dimensional array with the full path as a key

Array ( 
    [backend/frontName] => backend 
    [cache/graphql/id_salt] => 2BSdxbPSxQfMRyjZk5fxrPoxR2kaw5km
    [cache/frontend/default/id_prefix] => test_ 
    [cache/frontend/default/backend] => Magento\Framework\Cache\Backend\Redis )        

After further debugging I was able to discover that this flatterParams method is being called at least 8 times, with the same data to processed multiple times. This time it was not related to number of store IDs or extra configuration stored somewhere.

How to improve this? An idea in progress...

This is just an idea now, but I'll try to implement it in upcoming weeks. Here is a breakdown step by step how I see this being improved:

• when a configuration is being cached, data should be flattened to one-dimensional array during the save process. This way there is no need to flatten it once the config is read and processed by DeploymentConfig class.
• env.php and config.php are also loaded and processed multiple times, they should be processed only once and cached (imho). Looking for an inspiration on this.

Desired results:

Based on this very simple PR I've submitted to Magento Open Source:

https://github.com/magento/magento2/pull/38642

which is nothing fancy, it is just adding a simple cache (an array with keys) to store previously loaded and processed configuration.

This simple change has already reduced the load time of the configuration by half and improving the application initialization (for every request) by almost 200ms and for some pages / CLI operations / WEBApi request is has improved the page load time by almost 50%

The pull request which I've submitted on GitHub basically reduces flattenParams call from this:

to this:

Downsides:

Changes in this file https://github.com/magento/magento2/pull/38642/files#diff-078f6562fd96e2c5666acc3bc55022e383c2e9136268e8d2f99a9ce441db2370

which also gives 15-20ms improvement adds over 100MB to the memory required by PHP process.

I look forward for the community input on this, as this is a huge gain for everyone: developers, Merchants and even Adobe

Jakub Winkler

Prime Sorcerer of Strategy & Success @ qoliber