Measuring Usability: What Metrics Should You Track?
When establishing the key performance indicators for any site, they should always include measuring usability.
Measuring usability is often overlooked in favour of a focus on conversion. However, ignoring usability as a metric can have a significant impact on the long-term health of your business and ultimately will undermine users taking action.
See Also: Website Conversion: How To Measure Your Conversion Rate
Why You Should Be Measuring Usability
Usability underpins everything on a website. A website can be unattractive, un-engaging and even poorly written and it may still convert. However, if people cannot use a site, taking action becomes impossible.
A website can be unattractive, un-engaging and even poorly written and may still convert as long as it is easy to use.
Worst still, users have an extremely low frustration threshold when using a site. With so many competitors out there, why would a user struggle with a problematic website?
But good usability can provide a distinct business advantage too.
There is a reason why every Christmas I say to friends and families that they can have whatever they want as a gift, as long as I can order it from Amazon. I know Amazon, and so for me, it is easy to use.
My familiarity with Amazon makes me loyal to that site, and I am not alone. Users tend to be loyal to sites they find easy to use. That increases the customers lifetime value, allowing companies to pay more for customer acquisition so outperforming competitors in pay-per-click listings.
Like many, my bias towards buying from Amazon is down to the fact that it feels easy and familiar.
Customers will even pay a premium for a straighforward experience. That means if your website is easy to use you can potentially charge more!
How then do you know if your site is easy to use? How do you measure its usability?
See Also: How to Measure Engagement on Your Website and Why Bother
Measure Your Site’s Overall Usability
There are many ways to measure a site’s usability. However, most of the options are relatively time-consuming to implement, which means that they do not happen regularly. That makes it hard to track the effectiveness of site improvements over time.
Pragmatically most organisations need a cheap, quick method of measuring usability and that is where the system usability scale comes in.
The System Usability Scale
The system usability scale is a simple survey that asks users to express their agreement with a series of statements. Participants rank each statement from one to five based on how much they agree or disagree with it. Five means they completely agree, while one means they strongly disagree.
The System Usability Scale is a simple survey for measuring perceived usability.
What statements you use is of course, entirely up to you, but ideally, you should have ten. A typical sample list of statements for a website might read:
- I think that I would like to use this website frequently.
- I found the website unnecessarily complex.
- I thought the website was easy to use.
- I think that I would need the support of a technical person to be able to use this website.
- I found the various functions in this website were well integrated.
- I thought there was too much inconsistency in this website.
- I would imagine that most people would learn to use this website very quickly.
- I found the website very cumbersome to use.
- I felt very confident using the website.
- I needed to learn a lot of things before I could get going with this website.
How to Calculate Your System Usability Scale
Notice how the above statements alternate between positive and negative statements. For example, the first statement is positive:
I think that I would like to use this website frequently.
While the second is negative:
I found the website unnecessarily complex.
For each positive statement take the users score and subtract one. So, a score of four would become three.
For each negative statement subtract the users score from five. So, if a user scored a statement as a five, then the final score would be zero.
Once you have a number for each statements, add these together and times the total by 2.5. That will give you a score out of 100.
Admittedly all of that mathematical gymnastics is a bit fiddly, but it does give you an easy rating out of 100 for measuring usability. A metric that allows you to track improvements over time.
How to Use the System Usability Scale
Once you have the hang of it, the system usability scale opens up a world of possibilities.
For a start, if you use the standard statements as outlined above, you can get a reasonable idea of how usable your site is compared to other systems in general.
The average system usability score is 68 and anything above 80 means you are in a very good place. However, anything under approximately 50 and you need to be prioritising usability fixes.
The average system usability score is 68 and anything below 50 should be of concern.
But, you can also use the system usability scale to compare your site to the competition by asking users to rank multiple sites.
Finally, you can use the system usability scale to rate a prototype against an existing site, or to compare multiple design approaches.
The Limitations of The System Usability Score
Although the system usability scale is an excellent way of measuring a site’s usability and by extension the impact of usability on conversion, it is not perfect.
The system usability scale suffers from two flaws.
First, it can only identify how good or bad the usability of a website is. It cannot diagnose why it is succeeding or failing.
The problem with surveying users is that there is often a discrepancy between what they say and what they do.
Second, there is often a discrepancy between what a user says and reality. For example, somebody may describe a website as easy to use because they are familiar with it. Equally, they might say they will recommend that site to others in the moment of completing a survey, but not do it in reality.
These facts mean that we cannot only rely on the system usability scale. We also need to be measuring how users perform when completing critical tasks on a website.
How to Measure Usability of Critical Tasks
If you want users to complete a call to action on your website, taking that action has to be easy. But how do you know if signing up for your newsletter, buying a product or completing your contact form is, in fact, easy?
What Task Metrics to Measure
For that kind of specific action, the system usability scale will not help. We will need a different type of usability metric tied specifically to our calls to action. We will want to measure a series of factors. Factors such as:
- The time it took the user to complete the action.
- The number of users who simply failed to complete the task.
- Whether the user was able to complete the call to action using the most direct route.
- The average number of mistakes the user made when trying to complete the task.
Fortunately, these various metrics do not all have to be measured separately. We can ascertain all of those metrics by watching real users complete tasks.
How to Measure Task Metrics
You can track task metrics when running any facilitated usability test session as long as users are being asked to complete the appropriate tasks. However, because we are looking to gather quantitative data, testing with a more significant number of users is preferable as this reduces any one user skewing the metric with abnormal behaviour.
Because facilitated usability testing can be time-consuming, it may be preferable to carry out unfacilitated testing when seeking to establish metrics around critical tasks such as your calls to action.
In unfacilitated testing, the user is set a task to complete, and they then finish it without being directly observed or interacted with by a facilitator.
There are a number of tools that can help with unfacilitated testing of this nature.
One option is to use an application such as Lookback. Lookback allows you to carry out both facilitated and unfacilitated testingboth in person and remotely.
A tool like Lookback allows unfacilitated testing and the ability to watch each session back as a video.
Lookback will allow you to send a link to a user which when clicked will set that user a task and ask them to complete the work while being recorded. The advantage of this approach is that you can see users completing the task in video format along with a commentary of them explaining what they are doing and thinking.
The drawback, however, is that you are left to calculate the metrics by watching each session and recording the number of misclicks or whatever other metrics that are of interest.
One solution to this problem is to use a tool like Maze. Maze doesn’t record videos of user sessions or allow you to see the user completing the task. However, it does provide detailed analytics in regards to what those users did. That makes it ideal for task metrics, while Lookback is better suited to qualitative testing.
Making Sure Measuring Usability Happens
Using the right tool for the job is of crucial importance if you are to integrate tracking usability metrics into your workflow. If gathering those metrics is not straightforward then you will stop doing it, and that is a dangerous road.
We tend to focus on things we can see. If all we see are conversion numbers, then this will be the focus of the organisation.
Our tendency is to focus on things we can see. If all we see are conversion numbers, then this will be the focus of the organisation. The result will be the adoption of short term techniques to improve conversion (dark patterns) at the cost of long term success.
See Also: Dark Patterns and Aggressive Persuasion – 3 Reasons to Avoid!
If we start tracking usability, we not only ensure it gets the attention it deserves; over time we will also be able to demonstrate its value. We will be able to show that if we reduce the time it takes to complete a task, then there is a direct correlation with improvements in conversion.
Stock Photos from NicoElNino/Shutterstock