Flicking the switch - Feature Toggles and Cache Variations
The motivation
I have a shiny new feature that I want to try on our website, and I think it’ll encourage users to buy more things! But I’m not 100% confident that it’s going to work as expected, so I would like to test it out on a smaller audience first.
This is where feature toggles come into play.
In this article, I’ll be explaining the impact of feature toggles on a website’s cache, and will discuss solutions for this.
Feature Toggles
Feature toggles are a vital part of modern websites — allowing us to ship a new feature, style, or functionality to a subset of users before sharing it with a wider audience.
It can also be used as a ‘killswitch’ to remove features that may have problems out of your control, such as a feature that relies on a third party.
Having granular control over experimental or more fault-prone parts of the website is vital to the stability of your platform.
Cache
Another feature that most modern websites now make use of is a web page cache — a kind of store that saves identical requests to the site and returns the same response. This helps to reduce computation power and response times, but it does have its challenges when working with feature flags.
The main problem is that if the cache doesn’t take into account the feature flags being provided to a page, it will return whichever version of the page it has cached, regardless of the feature toggle. This is where we make use of the?vary header.
The?vary?header tells the cache which parts of the request make the request unique. For example — if we were to use a user’s?sessionId?in the?vary?header, we would create a new cache variation for every single request — and therefore might as well not have a cache, as every request to the website would require us to regenerate the page. In our case — we will want to use something like the?feature-flags?header key. This will tell the cache that for every request with the same URL and the same?feature-flags?header, it should be identical and therefore there is no need to regenerate the content, just return the value that we have cached.
Pause
After the above reading, I thought a nice light chapter would be fitting. Alas. Have a picture instead.
Ok, so what’s the problem?
Cache variations.
The number of cache variations we can store at any one time is very heavily limited by our CDN, for this example let’s say we have 50 variations available to us. As soon as we run out of space, the least-used variation will be replaced with a new one.
Another issue is that the number of feature flags affects the number of cache variations exponentially, instead of linearly.
For example, a request to?www.dunelm.com?might have 3 feature flags;?flag-a,?flag-b, and?flag-c. If these are all at 50%, so half the users will receive each page, then the number of cache variations looks like so:
领英推荐
So very quickly we will hit our cache limit. Especially when you consider that we have this cache for each page — so that’s eight variations of every page that we can store.
The less we are able to use the cache, the longer the average page load time is for the user, and page load times (along with other?core web vitals) are probably linked to customer purchases.
What’s the solution?
We have a multi-faceted approach to solving this issue so that we can make the most of our cache.
This solution leverages the tagging facility of our feature flagging provider. By adding tags to show which pages a feature flag is relevant for, we can filter out feature flags that will make no difference to the page a user is on. This is quite impactful, as the majority of feature flags are specific to one page only — so by removing them from other pages we reduce a lot of cache variations.
2.?MFEs
If you‘ve read?our previous blog post, you’ll know that we are rebuilding our site using MFEs, one page at a time. One benefit of doing this is that we can utilise a separate cache for each MFE — increasing the site-wide number of cache variations we can make use of greatly.
3.?Separating?cache-affecting?and?non-cache-affecting?feature flags
If your reaction to the above sentence is ?? then fear not, you’re not alone. In the next section, I’ll explain how we determine the difference between these flags, but the point you need to know here is that if the flag does not affect the?cached?version of the page it applies to, then we can avoid creating a cache variation.
What is a cache-affecting flag?
I’m going to use the age-old technique of the pizza analogy. Everyone understands pizza.
Let’s think of the ingredients as our codebase, the pizza oven as our server-side rendering location, the customer as the client side, and the toppings as feature flags.
Note:?Our pizza oven has a very cool ? magic ? feature called a?cache, which recognises a request for the same pizza and produces it without needing to cook it.?However,?it is limited to 50 different pizzas.
With no toppings (feature flags), we put the ingredients together and pop it in the oven (server), and the pizza that comes out (server-side rendered page) is always going to be pretty much identical. This means we can use our magic?cache?to produce the same pizza again and again without having to make a new one each time. This saves time and computation (sorry, cooking) money! Woohoo!
Now we have a customer who wants their pizza with pepperoni. Yum. With the?pepperoni?feature flag enabled. Crucially, the pizza that comes out of the oven?looks different. This means we cannot use our cached plain pizza this time, we need to create a new one and cache that too. That means the pizza topping is?cache-affecting.
Another customer comes in and says they would like a plain pizza, but that they would like rocket and parmesan sprinkled on top. Great taste! In this case, the pizza that we want to come out of the oven is?identical?to the plain pizza. The chef brings the pizza to their table, adds some fresh rocket, and sprinkles some parmesan. The?rocket?and?parmesan?feature flags are?not cache-affecting.
In this article, I have discussed what a feature flag is, what a cache is, and how we are using them together to enable experimentation of our site — all while boosting site speed as much as possible (by utilising the cache).
There were some pretty hefty topics in this article — so if there’s anything you’d like to learn more about, or any improvements you can see to the solutions I have suggested above then please reach out in the comments!