Auto-Tracking, Data Lake and Visualisation Are the Future of Digital Analytics

For the last twenty years or so tracking digital data has had much in common with carefully tended farms. Every data point is like a seed that is deliberately planted, nurtured, and maintained for subsequent analysis. While this precision-focused model can be effective there are a few drawbacks. It lacks flexibility and is resource-intensive in terms of cost and time to sustain.

What is auto-tracking

Auto-tracking solves these challenges in a simple way by collecting all the data points automatically without the need to specify them. That means collecting a lot more data in it’s raw format. This approach provides great flexibility as you give them greater definition retrospectively. For example, collecting all the clicks on your web pages will allow you to define which ones are “add to cart” clicks and which ones are “menu item” clicks based on the click data attributes. Data quality is guaranteed by having all the data points in raw format. You can even go back to redefine metrics without worrying about messy implementation and change management.

You might think collecting more data extends the implementation time for tracking. This might be true if you’re tracking individual data points in the traditional way. Auto-Tracking not only simplifies the implementation of analytics, but it can also enhance data quality. For the data to be used in replaying sessions with high fidelity, it must accurately capture events down to the millisecond, exactly as they happened.

Auto-Tracking however collects much more data than the traditional precision tracking. As big data technology continues to advance (especially the Data Lake), auto-tracking becomes much more affordable and efficient!

Digital Analytics Data Lake

A data lake is a centralised repository that allows you to store all your structured and unstructured data at any scale. You can store your data as-is, without having to first structure the data, and run different types of analytics — from dashboards and visualisations to big data processing, real-time analytics and machine learning. This is all used in businesses to guide data and insights based decision making.

In the context of digital analytics, because digital interfaces such as web browsers can be easily tracked in a standard way, it is the prefect use case for combining Auto-Tracking and Data Lake concepts to capture all the raw data while also enjoying the benefits of flexibility. High data quality is assured. It appears in this situation you can have your cake and eat it too.

When you’re tracking all clicks on your website there isn’t any need to worry about how the next release might miss some key button click data you might need. This advantage allows business operators to make swift decisions backed by a wealth of data to gain a competitive edge, especially in the post-Covid environment where agile is more important than ever.

Pros and Cons

Le’t summarise the pros of combining Auto-Tracking with Data Lake for digital analytics as discussed above:

• Flexibility
• High data quality
• Swift implementation process
• Low maintenance effort

What are the potential challenges with this approach?

A potential downside of the data lake strategy is the increased operational expenses tied to processing a larger volume of data. Nevertheless, these costs are counterbalanced by a decrease in maintenance expenses and greater flexibility with how you collect and use your data. The capability to amass all data in its unprocessed, raw state hastens the build-measure-learn cycle, empowering businesses to concentrate on learning and making rapid decisions supported by abundant data.

Another challenge is that when you have a massive amount of data it’s sometimes tricky to make sense of it. The way we solve this challenge at Insightech is through visualisation. Contextualisng data with visualisation means you avoid wasting precious time defining every data point. It also makes the data much simpler to understand by stakeholders across your business and facilitates data democratisation.

As an example, with a Click Map you have a much more accessible way to visualise what’s working or no on your website homepage for visitors coming through from SEO. Layering click and conversion data over your homepage enhances the visualisation experience making it easier to interpret the data - certainly much easier than if the data was presented in a table format.

Now that you’ve made your data visual and easier to interpret the next logical step is getting more team members involved. You may have heard about democratising insights. Visualising analytics makes this process of sharing data and insights widely across your organisation a much easier proposition. Everyone in the digital team can watch session replays to understand the user experience.

Session Replay is another effective way to understand the digital experience from the data points of individual website visitors. The precise nature of session replays are similar to watching your favourite TV show - only the show is about how your visitors experienced your website at that exact moment.

Building a Strategy for Data-driven Culture

While the Data Lake approach combined with Auto-Tracking provides substantial flexibility, it's still crucial to establish definitive measurement objectives, as they define your course of action. The versatility of the approach doesn't imply drifting without purpose or direction. By setting precise goals, we can exploit the data to reveal valuable insights even when events unfold as expected.

Lowering the barriers to gain insights from data is critical. Utilising visualisation tools like Session Replays and Click Maps makes the data more accessible and easier to understand, however this doesn’t mean everyone will become a data expert overnight. Data should be used as a tool to support specific roles within a team, not overwhelm them. I believe it's crucial to set the right expectations and build data literacy through practical workflows.

For example, marketing team can use Click Maps to understand user engagement on the landing pages, leaving more advanced analytical questions such as building data models to the data team. This thoughtful approach democratises data and breaks down data silos while simultaneously respecting the focus of individual roles.

Culture is defined by what you do. As you upskill individual team members to work with data you’ll gradually start to build a stronger data-driven culture. This is something I find is often overlooked by general advice. Much is said about the importance of being data-driven and using insights to make decisions but little attention is paid to how to build the data-driven culture progressively. By progressively upskilling the team and lowering the barriers to understand the data we can transition smoothly to form repeatable daily habits and strong systems to enable a data-driven culture.

I will continue writing my thoughts on building digital success with analytics as well as share the ups and downs as the Founder of Insightech. Feel free to subscribe to my newsletter and send me your feedback.

Here are some articles I’ve previously published which might be of interest:

• The Digital Analytics Evolution - A Founder’s Perspective
• The myth of AB “test everything”