User-Driven Testing with FullStory

Session replay is one of the most underused tools in the software quality arsenal. These tools, which are sometimes perceived as tools only for product managers, hold the secret to efficient regression testing.

When thinking about regression testing, the main question we want to answer is: “Will everything that my users did yesterday will still work for them tomorrow?”. How can we try to answer this question without knowing what our users are actually doing?

Many of us think we know better than our users how to use the product we build. This perception is also reflected in the way we write automated tests. We write a few “happy flow” tests, we write some negative test cases, and we write some edge cases tests inspired by our knowledge of the internal implementation details. These tests usually cover everything except what our users are actually doing - 

Session Replay to Automated Tests

Session replay tools capture and visualize how your users interact with your website or app. Giving developers access to witness that themselves is a great way to expose them to real usage patterns and enable them to create user-driven tests that focus on what matters most - our users.

One of my personal favorites is FullStory. It is extremely easy to install, has a generous free tier, and comes pre-configured out of the box with some cool pattern detection features like Rage clicks and Dead clicks. It also has the ability to capture the network layer and all of the API calls performed during the user session. This amazing feature opens up the possibility to automate the whole process of user-driven testing for the back-end.

Step-by-Step Guide for Automating User-Driven Tests

Start by signing up for a free FullStory account on this page.

After you created your account, embed the recording code snippet in your web app:

Go to recording settings and switch on the Ajax recording feature:

For user privacy reasons, FullStory will not record all of your API by default. Some of them might contain sensitive data that you wish to exclude from the recordings. Go to the privacy settings tab and create a URL pattern that matches all of the APIs you wish to record (you can also specify the request and response body fields you want to exclude).

Every user session captured on your website from now on will include all of the relevant API calls. Each page load event will show a link to download a HAR file listing all the API calls performed on that page with all the relevant info (HTTP method, body, header).

Using Loadmill, we can automatically transform this HAR recording into an automated test, which will replay the exact user behavior on that page.

Once uploaded, the Loadmill algorithm will create a parameterized script and write JSONPath extractors for the response values that are used in subsequent requests.

You can use this approach to create automated tests that reproduce real issues encountered by your real users. Having a test that reproduces the error is a great starting point for writing a fix and the best way to ensure this bug won't happen again.

This approach is also the fastest way to create extensive user-driven test suites that validate the impact of every code change on your users. This is critical because, for any non-trivial app, the possibility space for all existing edge-cases will always be too large to fully cover with tests. By relying on data to focus on the edge-cases that really matter, we can have the confidence to move fast without breaking things.