Data-Driven Decision Making: A Single Source of the Truth?

Data-driven decision-making: who doesn’t think it is a good idea? But it typically has a rough go in the real world of enterprise management, in part because the data itself often proves unreliable. For much of my business life IT has been tasked with building systems that could represent a single source of the truth. Unfortunately, that quest proved to be right up there with the holy grail and the fountain of youth—at best, aspirational, at worst, delusional. 

Today we have an opportunity to make a great leap forward, however, because for the first time in history we have broad access to high-volume data from a variety of sources that, when matched against each other, dramatically increase the probability of something like truth, and do so in a time window that is actionable. Not everyone, of course, has access to all the sources, so to kick things off let me present a framework of the possible, within which each organization can determine what its actual will be.. 

Systems of record occupy the bottom step in this maturity model, having been prevalent for closing in on three decades. They are the foundational business systems that consolidate around ERP and explicitly capture all the material transactions of the enterprise in a relational database from which they extract and present data to help manage internal operations.  We know them well and could not operate without them.

Systems of engagement, by contrast, are relatively new to the scene, be they in service to marketing automation, customer service, or sales enablement. Here again the transactions are explicit, but the data now reflects external market signals to help managers better align their operations with stakeholders outside the enterprise. Most enterprise IT organizations today are focusing the bulk of their attention at this level, the goal being to “get digital.”

The next level up, systems of intelligence, is just emerging onto the scene, having become highly visible in the leading disruptive enterprises who are currently eviscerating their competition. These are analytical systems that operate primarily on tacit rather than explicit data, extracted from the log files of websites, mobile phones, sensor-enabled devices, network traffic, and the like, supplemented with public data streams from Facebook, news broadcasts, and their ilk. Log data is tacit because it is typically being collected for syntactic rather than semantic reasons—meaning it helps with the technical operation of the digital system and is not engaged with its content. What systems of intelligence are able to do, however, is infer from these syntactic signals conclusions that have great semantic value, as, for example, a person’s propensity to buy, or click on an ad, or cancel a subscription, or perpetrate a network attack or fraudulent transaction. 

Finally, at the highest level, just coming out of the labs in most cases, are systems of autonomy, be they drones, self-driving cars, computer-navigated tractors, self-adjusting thermostats, autonomic factory equipment or the like. These are real-time operational systems that are also being guided primarily by tacit data coming from a wide variety of sources and sensors, including GPS, lidar, radar, sonar, accelerometers, barometers, maps, video, and the like. From all these signals these systems are able to infer position, attitude, and current condition relative to a given mission, and prosecutes a path through the physical world accordingly. 

These four very different kinds of systems generate four very different kinds of data in service to four very different purposes, as summarized below:

Now, given these four very different types of data sources, how should we be thinking about data-driven decision making? Generically it is clearly desirable, but what it entails varies considerably. Here’s how I see it playing out.

• Data-driven decision-making with Systems of Record.  

This has been the baseline for enterprise IT ever since we stopped calling it data processing and started calling it management information systems (roughly 1980) with it really coming into its own with the rise of business intelligence in the 1990s. The goal is to generate insights from the transaction data by transferring it to a data warehouse and applying a variety of tools to generate reports and analyses. Once these are made available to management, various teams do deeper dives to better understand and control their operations, generating insights that are input to a qualitative decision-making process conducted through staff meetings and quarterly business reviews.

One of the key challenges in this data-driven approach is a lack of a “single source of the truth.” The data is often fragmented, coming from disconnected internal systems, and typically supplemented with private data sources via spreadsheets that are not replicated elsewhere, all of which leads to conflicting representations of what is supposedly the same situation. Things are not helped by the fact that different factions within the enterprise often have different axes to grind, calling into question the integrity of the entire process. All this creates enormous frustration among decision-makers, and the most common way to resolve it is to default to the HiPPO (Highest Paid Person’s Opinion). This isn’t crazy—presumably that person is being paid a lot for a reason—but the likelihood that the data has been machined to manipulate their decision is high, and so also is the potential to drift toward increasingly bad decisions.

That said, in a prior era when demand exceeded supply for most goods and services, companies had time to recover from bad decisions since customers had few options to take their business elsewhere. With the turn of the century, however, the demand/supply equation has flipped, and now it is the customer who has the power to defect and the supplier who is left holding the bag. That makes it imperative to get closer to the customer, which in turn has been driving a massive and ongoing investment in systems of engagement. And that, as we shall see, creates a new kind of decision-making.

• Data-driven decision-making with Systems of Engagement

The goal here is to generate insights from digitally enabled contacts with the customer supplemented with information collected in the customer-facing systems of record. The data is by definition external and for the most part explicit, coming from call center logs, chat rooms, A/B testing, customer satisfaction surveys, or marketing automation responses, for example, supplemented by some tacit data from things like website clickstreams and abandoned shopping carts. As with systems of record, this data is fed into a qualitative decision process, but this time the decision-makers are more likely to be part of a cross-functional team since the interests of the customer quickly transcend the boundaries of any one function within the enterprise. These teams are normally staffed by middle managers with no HiPPOs in the room, meaning that the decision process is more collaborative, and the outcomes tend to be more tentative and provisional. 

The biggest technical challenge with this type of data-driven decision-making is integrating the customer-facing systems of engagement that are typically SaaS applications running in public clouds with data from the systems of record that are typically licensed software applications running in on-premise data centers. At the same time, at the business relationship level, there is a similar disconnect: customer issues rarely align with organizational boundaries, so there is an inherent complexity entailed in bringing any call to action to fruition. Cross-functional teams are empowered to look, see, and recommend, but they can’t then just pull the lever of change. Securing anything like a timely response in a hierarchical management system in nigh on impossible—hence the growing interest in a variety of more collaborative operating models: customer-centric design, agile development, and ongoing course correction—all designed with a bias toward action. 

Overall, this is a data-driven approach, but more often than not it is anecdotes and dialogs that are doing the heavy lifting. It yields significantly better results than just navigating by one’s systems of record, but it is highly prone to error, so much so that it builds in error correcting cycles into its core methodology. Thus, if one could find a more productive approach, there is plenty of payback. This observation has not been lost on a host of disruptive innovators, led by the likes of Amazon, Uber, Airbnb, and Netflix, who are raising havoc in a host of industries who never thought of themselves as high tech before. Now they must, and that’s what’s driving all the interest and investment in systems of intelligence.

• Data-driven decision-making with Systems of Intelligence

When we move to systems of intelligence as a platform for externally driven, customer-centric decision-making, the biggest difference is the shift from explicit data to tacit data. That is, rather than counting on people to report on what matters, and then count on other people to interpret those reports correctly, systems of intelligence interrogate log files that record the actual behavior of assets and people, and then use machine learning to extract signals of economic consequence. These signals are typically augmented with human expert input at the beginning to train algorithms to recognize actionable states and prescribe the appropriate responses. Over time, however, the algorithms become better than the humans, and at the point, you want them to be making the decisions, as they do today in algorithmic trading of equities, online fraud detection and security systems, and dynamic digital advertising. At the same time, you can’t allow them to go rogue, so these efforts must be constrained though policy, oversight, and human review.

Because it is early days, scarce data science expertise makes it challenging at present for many enterprises to invest in systems of intelligence, but that pressure is easing as the major cloud computing vendors—Amazon, Microsoft, Google, and IBM—are all making machine learning and artificial intelligence resources available on-line, supplemented with consulting help. In addition, the sheer data volumes involved are mind-boggling to anyone who grew up in a prior era, entailing wholly different approaches to where and how data is stored and analyzed. Again, however, the cloud vendors are more than eager to help. A more persistent challenge, on the other hand, is posed by data privacy. Where tacit data is being used to infer personal preferences, a boundary is crossed, and the issue is, what is the appropriate social contract to govern this domain? This is all new ground, and we can expect a broad range of responses. The immediate workaround is regulation through legislation, which we know from our experience with working out from under the financial crises of 2001 and 2008, will initially be awkward, painful and expensive. And finally, as if that were not enough to occupy our regulatory bodies, we have a fourth class of system looming on the horizon: systems of autonomy. These promise to bring to the physical world the astounding productivity improvements that algorithmic computing is bringing to digital processes.

• Data-driven decision-making with Systems of Autonomy

 I’m not going to say a lot about these systems except that, as recently as ten or so years ago, they were simply unimaginable. Now they look instead like the natural extension of systems of intelligence integrating with robotics. Who knew?

The data is tacit, and combines both internal and external signals to determine a machine’s location, velocity, and attitude relative to whatever environment it is in. The algorithms combine machine learning for machine vision and AI for navigational decision-making. This is still a wickedly hard computing problem, but with the wickedly smart people at work on it, it seems the truly persistent challenges will again be social, this time around safety, liability, and consequential ethics. For now the workaround is simply to sequester the bulk of the use cases to private property with restricted access.  

Well, that little digression didn’t take long, did it? Whew! At any rate, now we have a framework for digging into how our own organization might become more data driven. 

Let me begin by suggesting we have already extracted most of the value we can from the traditional systems of record approach. As with all highly valuable legacy capabilities, this means we are over-staffed in the function as historically conceived and understaffed everywhere else. How can we move our forces closer to the point of attack?

To start with, the increased deployment of systems of engagement gives us ready access to two options that were not really available before. First, we can get explicit data simply by asking users for it, and we can do so at scale, across any size population, in any time window, at a marginal cost of near zero. This is a spectacular gift which most organizations still lag far behind in adopting. Second, we can, if need be, take a “guess and then check” approach. This is the fast-fail lean start-up approach of agile development and minimum viable products, the one that puts an end to long planning cycles that do little to reduce actual risks and replaces them with low-cost experiments that can course-correct their way to innovative, differentiating outcomes. In both cases we are supplementing imagination with data on a fast-cycle-time basis. That is the big difference between the old approach to data-driven and the new—it’s no longer just about quarterly reviews of an annual plan; now it also includes weekly commits around strategic intents. The old mantra was “Measure twice, cut once,” a good approach whenever you are making a big bet. The new mantra is No big bets! Rather we want tons and tons of tiny iterative bets that add up to one big outcome. Time is the scarce resource, and every second spent hesitating is time lost.

Until your organization truly embraces agile decision-making, it is too early to transition your focus to systems of intelligence, so let us suppose in your case that it has, and therefore that it is time to do so. Now a new rule arises, one that can be socially disturbing: If it can be automated, it should be automated. That is, wherever they can be made to work, algorithmic systems are cheaper, faster, and better than human systems—full stop. They don’t start out that way, to be sure, but in the end IBM’s Deep Blue defeats Gary Kasparov, IBM’s Watson knocks off Ken Jennings, Google’s Deep Mind defeats Fan Hui, and Amazon’s Alexa knocks your socks off (and then orders you a replacement pair). More immediately for the rest of us, any enterprise that has successfully deployed systems of intelligence at scale, regardless of industry, has dominated its competition in record time. If one of these barbarians is at your gate, you have no option but to move, and move fast.

The key here is to target your most critical moments of engagement with customers and prospects, determine the tacit data signals that give you your best insight into how that moment is unfolding, do whatever it takes to get access to that data in real or near-real time, and set hot shot data scientists to work at translating those signals into actionable responses. You are not replacing in humans here by any means, but you are promoting them to a new status. They are no longer your primary source of data; now they are your first or second wave of response, one that can bring imagination, experience, empathy, and pattern recognition to situations that cannot be resolved algorithmically. Without data, however, these folks are flying blind, doing their best to pick up the pieces on the fly. Armed with data, they can look like water-walking wonders. But only after you have deployed the systems of engagement necessary to collect that data and connect them to the customer.

So, to wrap, four types of data, for types of data-driven decision making, aligned around a digital systems maturity model. Your assignment, should you choose to accept it, is to identify your current state, target your future state, and leverage this framework to build and inspire a coalition of the willing.

That’s what I think. What do you think?

Geoffrey Moore Mailing List

______________________________________________________________________

Geoffrey Moore | Zone to Win | Geoffrey Moore Twitter | Geoffrey Moore YouTube