How to make sure you capture the right data for your product?

In my experience, I saw the same pattern in a lot of projects over and over again, where tracking implementation often times ends up in being a part of the development team responsibility, which leads to developers being those who need to come up with a tracking concept and data collection points. But a lot of confusion comes when teams are not capable to provide meaningful data after tracking is being implemented, just finding out that some key information is missing, or captured in the wrong way for providing any insights.

One of the additional issues, which come along with associating this task to the development team primarily, is the underestimation in resources required for implementing the tracking, so the developers spend far more time to find out what exactly needs to be done, instead of focusing on just the implementation part, the thing what they are biggest experts in.

To utilize the power of the web and mobile analytics in the decision-making process, companies with dominant marketing teams and development teams need to have someone in between, who can make sure that the requirements coming from the marking, sales, and management teams are translated correctly to the actionable items for the development team.

Business requirements as the foundation for data collection

One of the first requirements to making sure the right data will be collected is to create a business requirement document first. Most business requirements are already available since the development roadmap needs those requirements for orientation. It's just important to keep in mind that business requirements are no to-do lists or project plans. Business requirements specifically provide information around the most important parts for the business to maintain growth, being profitable, or delivering a quality product.

As an alternative, instead of business requirements, another type of documents can be utilized, like storyboards, feature maps, or user flows. The way to work with those types of documents is a little bit different, but the fundamental idea of outlining the parts which are essential for providing the information if the business can achieve the goal or objectives stays the same.

The correct way to look at the KPIs

After defining the business requirements, the next steps would be to identify the key performance indicators (KPI’s) which can reflect if the business is moving in the correct direction. The most common mistake here is, that by defining the KPI's, the team ends up accumulating all the possible metrics and dimensions that can be collected, instead of focusing on what would provide meaningful insight about the actual performance. Some examples for wrong KPI’s world contain things like: “number of sessions”, “number of conversions”, “clicks on the specific item”, etc. None of them provide actual information if the change has a positive or negative impact on achieving the business goal.

A correct KPI must solve the minimal requirement of providing information about the change between two periods. Translating to the bad examples above, the correct KPI could be something like “growth of user engagement”, “growth of customer retention”, “growth in conversions”, etc.

After preparing the KPI list, make sure to verify that the list contains only those indicators, which can be reported as a simple two-color indication, with “red” color for performance is going down, and “green” for performance is going up. In most cases, the KPI list contains only a few items and is rarely longer than a list of 10~15.

One of the examples for KPI vs metrics would be an indicator that determines if a sales funnel is performing well. Obviously, the first thing which can come up to mind is that the performance of the sales funnel is basically the number of sales at the end. But to draw meaningful insights the performance indicator needs to provide information about the growth of the ratio between the people who start the funnel and those who convert to customers, as a comparison between two time periods. Along with the assisting information about how each step of the funnel performs, those KPIs would provide enough information for actionable steps to optimize the process.

Solution design for capturing the data

Now, once we have created a KPI list, the analytics team can translate those into metrics and dimensions. The result of this process often is called a “Solution Design Document” and it’s an important part of the tracking implementation process which summarizes all the parameters you need to collect for your reports. It can be as simple as an excel spreadsheet up to a more complex structure by using complex visualizations. In all cases, the “Solution Design Document” needs to outline which parameters need to be collected on specific pages, screens, or actions.

Translating the solution design into actionable items

Once you develop a solution design document outlining all the dimensions and metrics, it's time to create a guideline for your development team. Mostly this document is called either “Tracking Concept” or “Tagging Guide”. On this particular step, you can’t avoid having someone in your team with some level of expertise about your analytics system, because each analytics system has a different way of capturing and processing data.

One of the examples would be it's the comparison between Adobe Analytics and Google Analytics data collection, where Adobe Analytics provides a lot of flexibility on how the data can be collected and Google Analytics being stricter about the form the data should have.

As an example, Adobe Analytics supports the so-called “list variables”, which allow sending multiple values in the same parameter, separated by a delimiter like a semicolon. In the background, Adobe Analytics split those values, separating them and reporting each one individually with for example the number of occurrences. In the case of Google Analytics, each value is required to be collected separately, with a separate request, to analyze the number of occurrences individually. In other words, where Adobe Analytics would just require one single request to collect 10 items, Google Analytics needs to send 10 different requests, each item individually.

In summary, the first step on the path to capture the right data, is to focus on business requirements and translating them to reasonable KPIs rather than start thinking about the solution by listing metrics. Once defined, the process moves into the next phase, which is creating a Solution Design Document as a foundation, before the actual guideline for the development team can be created, the Tracking Concept.

The reason behind following those steps roots in having a process that is repeatable no matter which type of product you have. Following those steps contributes to implementing tracking effectively for your type of business, rather than blindly following the standardized solutions.