Improve your Web Analytics Data Quality
Do you trust your web analytics data?
Whether it has been a week, a month, or a year after your website analytics strategy is implemented; how do you know if the tracking is still accurate?
Most likely you don’t and you are not alone.
Data tracking monitoring and data governance in digital analytics is only beginning to gain visibility. Unless you test every single component of the implementation, every day (which is by no means practical) you cannot have full confidence in your data quality.
Why do Analytics Implementations break?
Analytics Implementations break for a variety of reasons:
- Inadvertently breaking dependent components (one change breaks another)
- Changes in the functionality of the website itself
- Tags were removed by accident or duplicated
Broken implementations result in incorrect data, which is known to:
- Erode organization confidence in web analytics
- Hinder analysis and decision-making
- Prevent proper execution of optimization opportunities
- Lead to bad decisions and result in lost opportunity value
No Trust = No Data-Driven Decisions
Of all of these issues, one of the most critical is the loss of confidence in your data. If you experience this in your organization, it is a huge step backward and you will have massive work ahead of you to restore confidence. If your organization can’t trust the data, they won’t make business decisions with it, and thereby your analytics implementation becomes worthless.
It is common that companies request to work with our team because ongoing data quality wasn’t part of the project plan. Even with a well-executed analytics implementation strategy, if the effort ends immediately after the tags are placed on the page, you are at risk!
Web analytics implementation is continuous because your website isn’t static — you change design, add new functionality, and improve conversion flows.
How can organizations avoid data quality issues that lead to these problems?
Web Analytics Quality Assurance (Manual)
One method to avoid these issues is through proper quality assurance (QA). By verifying if the implementation is working according to the specification, you are able to increase confidence in the data.
There are various tools and techniques to QA your web analytics implementation, such as:
- Google Analytics Tag Assistant
- Adobe Analytics Debugger (formerly DigitalPulse)
- Charles Proxy
Before publishing new or changed code into production, the analytics developer confirms it is working as expected by inspecting the hits sent to the web analytics platform and also confirms the reporting interface matches expectations.
This level of QA is a mandatory step during the implementation process, but it isn’t enough. The problem with manual QA is that it is just a snapshot in time, and not a continuous process.
Wikipedia: Quality assurance (QA) is a way of preventing mistakes or defects in manufactured products and avoiding problems when delivering solutions or services to customers.
Web Analytics Quality Assurance is:
- A manual process (prone to errors)
- A snapshot in time
- Slow & time-consuming
Web Analytics Regression Testing (Automated)
Regression testing is a type of (often automated) testing that aims to ensure that changes to the website (including tagging changes themselves) have not adversely affected the data. The goal is to ensure that what was implemented and previously QA’ed is still working after changes in the environment have occurred.
In comparison to Web Analytics QA, regression testing in web analytics is not a common practice. This is often because it is perceived to be expensive to implement and the consequence of an implementation failure is (incorrectly) considered as insignificant to the business.
In fact, it is a huge issue for the business when you destroy confidence in data; it is difficult to get that trust back. This affects organizations of all sizes if they don’t have the proper data governance in place. Making decisions on incorrect or incomplete data has a major direct and indirect cost to an organization.
To avoid these data quality issues, regression testing should become an integral part of the web analytics strategy and at the same time as the implementation is crafted (but start as soon as you can).
How it works: At regular intervals, and before each change is published to production, a process runs all automated tests and verifies that the web analytics implementation is working as expected.
Regression testing provides stakeholders the confidence that their web analytics strategy is working like the day it was deployed.
Wikipedia: Regression testing is a type of software testing that verifies that software that was previously developed and tested still performs correctly after it was changed or interfaced with other software. Changes may include software enhancements, patches, configuration changes, etc.
Web analytics regression (data quality) testing is:
- Automatic
- Continuous
- Fast
Tools for Web Analytics Regression Testing
As previously stated, the cost (resources and technology) of implementing proper regression testing on your web analytics data can be high. There are data quality auditing tools that can streamline the setup — but be warned that all tools will require some level of manual setup for more advanced scenarios (for example measuring clicks on a specific button rather than measurement of simple page loading hits).
These regression testing (aka data quality testing/auditing) tools range from enterprise-grade tools (paid) to open source tools (free) that can be heavily customized to meet your needs.
Paid Enterprise Data Quality Auditing Tools
The two most popular enterprise analytics data quality auditing tools are ObservePoint and Hub’Scan.
Both of these paid tools are fantastic and the Blast Consulting Team regularly use these with our clients. They provide automated scanning and execution of scenario-based testing and then provide alerts of any issues found — they each have a unique feature set that should be explored. There’s a lot of great features in these enterprise tools.
You can get up and running on these tools very easily (and likely find data quality issues almost immediately).
Open Source Enterprise Data Quality Auditing Tools
Another option is to explore open source packages. I recently developed an early version of a tool named WAUnit.
Features & advantages of WAUnit:
- Developed in Python
- Leverages Selenium scripting
- Provided code supports Google Universal Analytics hits
- Free open source software (download from GitHub) that you can use and customize (hopefully contributing to our project to push it along further)
Warning: WAUnit is a technical solution that requires knowledge of Selenium, Python, and general scripting.
Watch this video demo of the WAUnit data quality testing tool to learn more.
Next Steps?
- Talk to your analytics consultant about automated regression (data quality) testing for your web analytics.
- Explore the tools recommended in this post to find out what works best for your organization.
- From day one, plan for your analytics implementation to match your strategy and avoid the scary road of restoring trust in your data quality.