Data Shifts Power Within Organizations

A major challenge in going more data-driven in organization has less to do with data itself, and more to do with the ability to manage the dynamics that emerge as decision makers look at data as an input to decision process.

I have a particular kind of power shift in mind. I am not referring to the process of democratization of data, which we will discuss in a future article, but rather one due to the emergence of a new powerful organizational entity, The Scorer. The Scorer derives its power from data and its ability to judge both past and future decisions. The Scorer provides alternative decision paths taking away some agency and control from existing decision makers. In this article I explain why and how this happens, and how the process could be managed.

I felt the need to write this note because of a series of experiences and conversations I have had in the recent past in organizations that invested in data analytics to improve their business. The Scorer, in some shape or form, will show up early in the journey.

Decisions are Made Under Uncertainty

The world is a complex and dynamic place. Expensive decisions such as whether to open a store in a given neighbourhood or not and launch a product are frequently made in the presence of high uncertainty. The uncertainty has many dimensions including time, space, product, market, and organization. Most of the decisions are forgotten as the organization moves forward, and any mistakes are absorbed. Some organizations have constructive mechanisms to review the decisions, learn from them, and adapt. As people switch between organizations frequently, these lessons are also lost.

Data Provides Institutional Memory for Every Decision Ever Made

Organizations have always collected data, and but we have only started applying it seriously about ten years back. With more data being captured, and the cost of access dropping, it has become easier to go to data to evaluate past decisions, course correct those in progress, and set up mechanisms to evaluate future decisions. As technology and cost limitations on storage and access are removed, every single action taken by the organization is recorded and available for any purpose.

Decision Maker is Being Implicitly Scored

The decision process, the inputs, the choices made, and the outcomes are all being captured as activities and notes in the enterprise applications we are deploying today including email, task management, software development, and application software. We designed these for efficient execution of the decisions. But the same systems also, as a side effect, capture context and decisions more rigorously and in larger volume than ever. An audit trail is being established.

In theory one could look through them to assess the quality of every decision made.

Human intuition is limited, our mental models are biased, and our decisions are error prone. Increasing data resolution enables discovery of opportunity landscape, and helps find global optima or multiple local optima for decisions. In the past when any adhoc, economically unsound, or conflicted decisions was made, decision makers believed that their decision or process couldn't be questioned. These decisions are now up against the alternative omnipresent risk-mitigated decisions that data within their organization suggests. So the cost of not going to data for inputs and/or getting it wrong in the face of data will increase significantly as time passes.

The “Scorer” Is The New Power Center

Organizations cannot go data-driven without introducing a new stakeholder at the table - The Scorer. This entity enables, reviews, and influences decisions. Directly or indirectly it boils to an individual or a team. CDO/CIO is the role and definition that comes closest. In general it is anybody who organizes information and brings to bear statistical and systems tooling for scoring decisions. 

We know a few things about the Scorer and scoring process. First, the scoring process is not deterministic. There is significant judgment involved in interpreting data and statistics. Second, the scoring process is laborious and involves building systems, models, and experiments. The scorer will make judgments about where to spend resources. Third, the Scorer also develops independent new, and sometimes deeper, understanding of the business. Last, the Scorer and his/her team live in a mathematical modeling world that is not easily accessible to most people including the best managers. Reviewing the Scorer’s judgments is not likely to happen frequently or conclusively.

Given the above, the Scorer has choice of what to focus on, which methods to apply, and how to interpret without a strong review process for any imperfections. This adjudicating ability is ultimately the source of power.

Organizational Responses

There are a number of ways the introduction of a Scorer into an organization could play out:

1. Constructive Tension - Constructive but mildly adversarial relationship is established between the Scorer and rest of the organization with agreements on knowledge sharing and evaluation processes.
2. Cede control - Going with the Scorer’s recommendations will reduce the chance that the Scorer will blame the decision maker for poor decisions in future. This also reduces role of the decision maker and ignores the tacit knowledge that he/she carries.
3. Discredit - Decision maker could use adversarial techniques, especially poor quality of data, erroneous domain knowledge and politics, to prevent effective functioning of the Scorer
4. Coopt - Decision maker could use standard organizational mechanisms to co-opt the new function so that Scorer role and function is not effective. This is status quo.

Organizations that don't explicitly manage the process could see any of all of the negative dynamics.

Third Party Scorer Is A Minefield

Much of the above discussion assumes that the Scorer is part of the organization, and has a recognizable title such as CDO. Organizations that outsource analytics may find themselves in a position where the Scorer is outside the organization, and has a different set of incentive structures. The Scorer, in this case, has the power to influence decisions without having skin in the game. The Scorer is also developing significant competitive knowledge about the business through data.

This setup could go wrong in many ways. First, the Scorer may choose to not share or walk away with the new knowledge created. Second, the organization gets more dependent with every passing day on a third party entity, and the third party may develop monopolistic tendencies. Last, as decision makers cede control, the Scorer may make more decisions every day, and organization may lose valuable experiential knowledge and decision making skill.

What Leaders Should Do

Leaders should recognize that making decisions with data is a skill that will develop over time with practice. We can make a number of recommendations to get this right:

1. Scorer Team and Incentives - Selection of people for Scorer team and alignment of incentives should be given the highest importance. People should have the highest integrity, competence, and should have no hidden agendas of their own.
2. Empathetic Transparent Culture - Space should be created for learning with data, and individual decision makers at all levels should be given resources to extend their current team and capabilities. This extends to Scorers as well. Process and technical norms should be established to avoid perception and actual arbitrariness, and build trust in the institution.
3. Knowledge Recovery - The organization should always ensure that new knowledge discovered from data by Scorer is always accessible and proactively disseminated throughout of the organization.

Image source: https://www.flickr.com/photos/124961070@N02/14561580972