Client Side A/B testing vs. Server Side A/B testing: Pros and Cons

Okay – this debate has been going on for years now.

In 2016, Optimizely introduced Full Stack (which they have renamed as Feature Experimentation in 2022) gave a wider range of visibility to the ability of server side testing option.

However, they were not the first to allow customers to run server side testing. Many other testing tools like SiteSpect, Adobe Target and in house development team were capable of running server side testing long before that.

At the end of the day, the fundamental principal remains the same: ability to test a hypothesis, to make informed decisions, to ensure that the data is at the forefront to make changes. The way of testing is where it differs. Before we dive in with the pros and cons, let’s just take a look at the basics:

What is Client Side A/B testing:

You simply add a few lines of JavaScript code to the website. And then through the tool’s interface, you develop the experiments, create all the necessary targeting conditions, run the experiment. The experiment lives within the testing tool and gets executed on the customers’ browsers.

What is Server Side A/B testing:

You create the experiment or the variation code as part of the original code within the code base. You wrap the changes up with some kind of identifier (or flag) to show that this is part of the code that needs to be executed as a variation. If a user is bucketed to the variation, then that piece of code is executed from the server and then the user will see the variations.

As you can see, from an architectural standpoint, they are completely different. And both of these methods have their own pros and cons.

Client side A/B testing benefits

Let’s start with the pros of client side testing:

Quick and easy to implement: You will need to add a few lines of JS code throughout the website. And once they are (correctly) in place, that’s it from the testing tool implementation perspective

A/B testing variation code kept separately: All the variation testing code is kept and served from the testing tool. You don’t need to worry about the code being changed on the backend server or the main code base

Allowing wider group to run experiments: As the A/B tests are created on the experimentation tool itself, the marketer are able to launch small experiments fairly quickly. For example, changing the copy or adding a new copy. With client side testing, your backend development team can focus on BAU features that needs to be worked on, whilst a different team without have the access to the codebase can develop the experiments for you.

Velocity: As you are able to develop experiments within the testing tool with multiple teams, you are able to increase the testing velocity by getting additional engineers who are not necessarily part of your core development team

User interactions as metrics: easy to setup with (most) client side testing platforms – on action, trigger an event

Efficiency with QA: when the variations are created with the client side testing tools, the QA engineers can easily launch the experiment with the QA parameters to make sure that everything is working as expected and easily get the bugs fixed before pushing the tests live.

Testing type: anything that is related to the design and functionality (that does not require backend database update), you can use client side testing. Whether it is full page redesign, changing the funnel or adding a new section – you can test them out.

Client side A/B Testing drawbacks

Load time: does not matter how small the script is, there is (even if in milliseconds) an increase in page load time. As the script needs to be loaded from the client browser, that is something that can’t be avoided with client side testing

Winner implementation: when you are creating experiments that are not simple copy changes, when you find a winner, that needs to be implemented to the main code base.

Testing limitations: whilst with client side testing, you can pretty much test everything related to the design and most functionalities, if a variation requires saving or updating anything to the backend database, you are stuck.

Benefits of Server Side A/B Testing:

Now let's look at the pros of Server Side A/B testing and where they are good at.

Code implemented directly on the server: the variations are implemented as part of the main codebase. As a result, when you find the winner, you can simply stop and remove the previous code and keep the variation code as part of your codebase. No need to have another step to implement the winner

No additional impact on load time: As this is built from the server side, when the user’s browser is requesting a page, the variation is served straight from the server rather than manipulating the DOM elements on the browser

Limitless testing: there is no limit as to what you can test with the server side testing. You have got the freedom to change and test a new search algorithm, new pricing content, new packages – there is no painted door tests.

Same team working on experimentation: your development team who are working on the BAU, will also be working on developing the experiments. They will be working on the same codebase, making sure that everyone is aware of what is being changed.

Drawbacks of Server Side A/B Testing:

New Workflow Implementation: well, with the server side testing, the initial piece of adding the library to the existing codebase and making sure that everything is working as expected to develop, launch and manage the experiments can take time and requires a new workflow for the team.

Unit testing and QA: with most server side code changes or creating variations, the developers need to implement unit tests and additionally, QAing the variations that are built on server side can take longer as it requires additional launch requirements to only access the serverside testing variations

Reduced targeting criteria: whilst on the client side experiment, you can setup a targeting conditions based on user interaction (e.g. only start the experiment when the user clicks on Add to cart button), with server side testing, there are some limitations with those special targeting conditions.

Limitation with metrics: depending on what you would like to measure, there could be limitations with regards to what you can setup as the metrics. For example, if you are trying to measure interactions with a tabbed area, it could be tricky.

So - which one should I choose?

Choosing between the client side and server side testing – is completely up to you based on what your requirements are.

If you want to move fast with an experimentation program and don’t need to rely on the internal development resource to develop each and every experiments, client side A/B testing could potentially be the best option.

If you have got complex experiments that require changing the backend data, and you have got the resource and budget, server side A/B testing could be the way to go.

There is however, a third option - a hybrid option to run both client side and server side A/B testing. I will talk about that next and how some companies are successfully managing to run this hybrid option.

The ultimate thing is to make sure that you understand what the business goals are and what you would like to achieve form the experimentation programme.

I am sure I have missed other pros and cons – feel free to add your comments as to what your experience is with client side testing or server side testing and also if you are using a hybrid of client side and server side testing.

#conversionrateoptimisation #abtesting