Digital Analytics' War of Independence

Still reflecting on #MeasureCampSTHLM, Piotr Gruszecki mentioned in his session that silos are a byproduct of managerial rivalry. Data has changed how managers lock horns with each other. At companies valuing data and how it can help them make better decisions, the managers who learn to wield data have the advantage over their peers. The next move is for more managers to value and collect data they won't share until showdown. That sounds like the very definition of data silos: data that should be shared company-wide but protected to maximise its potency.

More significant challenges ensue when metrics with the same name exist in two or more silos but with different values. This can only happen when different teams define these metrics differently. The data will change with other definitions, but whose definition is correct? At the root of the issue lies self-serve analytics. Companies select a data vendor and let their departmental heads loose on using the tool. Each department defines its own metrics. Some may document their definitions, others may not. A more significant cause for concern is when politics influence these definitions.

An excellent example of this is how countries calculate inflation. Imagine that you are the incumbent political party in power, and the president said they will drive the inflation down. How can you do it? Inflation is the average of prices for everyday products and services. Now, everyday is a very subjective term. Depending on what you pick for the average, you can generate an inflation number that will come down and make the president look good. However, many products and services that are just as, or perhaps even more, common as what is in the inflation basket of products and services see their prices increase. The result is a wide gap between the official inflation and the one people experience. Definitions are crucial and will change to suit the political message.

Another example is joblessness figures. Your president promised to tackle joblessness. Here, you could introduce the slightly adjacent concept of job seeker. If, as a jobless person, you cannot demonstrate actively looking for jobs or fail to meet a threshold, you are not a job seeker. You may be unemployed but are not actively seeking a job. Voilà, job seekers' numbers are down, and to produce lower numbers, you have to increase the threshold that qualifies you as a job seeker. Officially, the employment figures have improved, and perhaps your country has reached full employment. Yet, people know people without jobs all around them, with two or three jobs to make ends meet.

Definitions matter, and politics will shape them. Every country will have its definitions, making comparisons difficult. At the scale of a single company, the departments may be in a position where they define their metrics. Marking your homework will always be an issue, and you exhibit signs of self-serving bias. Metric departments share will invite comparisons, and people will assume that a company-wide definition applies. What if each department cooked a self-serving definition and another one, or even more, each has its variants, all with wildly different numbers? How do you trust and generate business value from data in such a scenario?

Meanwhile, down in the trenches with the subject-matter experts (SME), the people are doing what their head of department tells them. The CXOs suddenly realise that data is not delivering business value and reassign the budget to AI initiatives. Who carries the most risk of losing their jobs in such a scenario: the departmental heads who concocted self-serving definitions of their metrics or the SMEs?

The rather strange metaphor, oh how I like my metaphors, that came to mind when I decided to pick this topic is colonialism. As a thought experiment, let's imagine a world with four significant countries fighting each other for dominance and plenty of neutral countries. The four countries are running low on resources such as oil, common and rare metals, building materials, and food. Colonial history has taught us how the leading countries will exploit the resources of the neutral countries.

Many may support the idea of the colonising country prevailing in the colonised countries. At home, they might see some economic development trickling down. In the colonising country, there could be opportunities, too, such as jobs. Instead, the scenario unfolds differently. The leading class in the colonised country keeps the wealth in its hands instead of letting it trickle down. In the colonising country, jobs are scarce, the harmonious cultural blend is slow, and racism is rampant. Unrest becomes increasingly frequent, eventually leading to a war of independence.

As digital analytics practitioners, we serve customers despite rarely meeting them. The data we work with expresses their behaviour, enthusiasm, and frustrations. We seek to improve their experience without passion or prejudice to make them prefer our brand over our competitors. However, we support stakeholders who see each other as rivals rather than serving the customers directly. The raw data we collect goes through a refinement stage that produces the metrics we see on the dashboards. The refinement could leverage self-serving rather than customer-serving definitions to support the departmental heads' need to stand out over their rivals.

The digital analytics colonial era may have started around 2010. The business recognised the business value; however, the stakeholders and departmental heads wanted control over the narrative. Their business case was that we, the digital analysts, were external hires, which meant we lacked domain knowledge and business acumen. Their solution was to turn digital analytics into a service function limited to extracting raw data. Gone was the mandate for producing actionable insight and the appeal of impacting the business.

A centre of excellence addresses self-serve analytics. A centralised team neutrally defines all metrics, especially company-wide metrics. By appointing someone internally to manage the team, you address critics about the team's lack of domain knowledge and business acumen. The company can now compare departments, knowing that the same definitions apply across all departments. I call this struggle for an independent view the digital analytics war of independence.

The stakeholders should not be in a position to cherrypick the data that will help them create a narrative about their department's performance - it's no different to marking your homework. With that loss, there is also no more need for data silos. With AI grabbing the headlines, I see how stakeholders may feel digital analytics is another tale of sour grapes: "We can't define or pick our metrics? Oh well... digital analytics is irrelevant now. I'll try that tactic with AI instead!"

The challenge of regaining the right to generate actionable insight remains. When you leverage the IKEA effect, data analysts and the stakeholders co-create actionable insight. The former brings their expertise with the data tools stack, and the latter brings that domain knowledge and business acumen, informing what matters to the business and leading to the era of digital analytics interdependence. Data analysts can deliver business value again by collaborating with the stakeholders and offering them a superior alternative to self-serving metrics and data cherrypicking.

#MeasureCamp #DigitalAnalytics #WAWCPH #CBUSDAW