Single Source of the Truth: A Love/Hate Relationship

"Thank you for joining us at DGA (Data Geeks Anonymous), my name is Chris and it has been three hours since my last buzzword."

An Apology of Sorts

I'll freely admit I have flip-flopped on the concept of the single source of the truth. Several years ago, I was heading up fledgling information governance program at a large organization that could not get out of its own way. The IT department and the Business had a confrontational relationship and the cost of that was a lack of understanding. To the people trying to make sense of it, there were multiple truths available to them. At that time given the organizational pressures, my best effort at rationalizing it was to push an agenda of democratizing data. (Eliminate the bottlenecks, create a culture of innovation, etc.) Hoping that if we went through the initial growing pains, the Wikipedia effect would take over and the truth would rise to the top. I went so far as to give presentations at well-respected conferences that the "concept of single source of the truth was dead".

In my defense, I'm sure the concept of democratized data (as defined above) is NOT a terrible idea. I just don't think this organization was ready for it. Sometimes a flock just needs a shepherd. That said, it wasn't a complete failure - many people learned many important things, including myself.

Arguments were won and lost, but the true victory was that the conversations were occurring.

The bitter irony of this is that we were truly working towards achieving the single source of the truth, the difference is that we were allowing for a certain data Darwinism rather than simply promulgating what we believed was fact from an ivory tower built on ones and zeros.

Fact and Context - AKA Plenty of Love to Go Around

As with anything, what's most important is the context. Certain data is fact. A transaction occurred at the intersection of all of the nouns: between entities, at a place, for a thing, often in exchange for a certain amount of currency. The "who, what, where, and when" shouldn't be debated. Modern analytics though are getting bolder, and are pushing the question of "why and how".

Why did it happen? Why didn't it happen? How could it have been more/less?

Think about this example for illustration. An organization has a number of employees. Each employee has a name, hire date, salary, etc. None of those facts should be disputed. A report of headcount, and salary spend should be 100% accurate 100% of the time.

But wait, there's so much more. If an employee leaves the organization, how could we have predicted that? How can we look at a chain of events (potentially dozens of factors both quantitative AND qualitative) and determine if it is a pattern that can make the organization better at retaining talent. This is where the concept of the single source of the truth starts to get a bit murky.

If the last few years has taught me anything, it is that the perception of truth is personal, relative, and subject to interpretation. To succeed, we must embrace the chaos and attempt to make sense of it.

This isn't a few pages torn from my diary where I was quietly apologizing for trying to get people to abandon the quest for the single source. I still believe in the power of truth, and never thought it was truly "dead". That was just an attempt at an attention grabbing headline. There will always be strength in fact, and easy victories in statistics. Math alone does not tell a story though. Humans are storytellers, after all. The "how and why" is the differentiator these days because we finally have the computing power to understand it.

To be successful in the modern analytical environment, we need to be architects of context.

So a moral of the story (story pun very much intended), is that the single source of the truth is an important component for fact based data. That said, an enterprise platform cannot live and die on one truth systems alone. The unstructured, the supporting information, the wild hail Mary passes deep down field are becoming just as important to our collective success.

How to Get There (AKA - The Pitch)

A while back there were a few relevant toolsets in the Business Intelligence space, and then as the market grew, the competition grew in kind. Some of the larger, more established products were a little like the ill-fated Titanic (large, luxurious, but with a small rudder). The smaller ships could navigate the iceberg field of rapid innovation, but they lacked the comfort of their predecessors. The good news is that in this story the ship doesn't sink. For the top tools, the course is corrected and a new 360-degree jet pod has been installed in place of the rudder. When you are looking at a solution - ask yourself the following questions:

• Does the platform have true enterprise distribution and scalability? Can it get the information to the hands of the people who need it without massive amounts of intervention?
• Does the platform fully integrate pixel-perfect reporting for your operational needs along with exciting visualizations, spatial data, and true self-service capabilities?
• Is the AI and machine learning capability of the tool strong enough to make sense of the volumes of context needed to bring your program to the next level?
• Does the tool have a governance and catalog that can integrate your fact with your context?

Some of the market is incredibly inexpensive but doesn't check many of the boxes above. Some of the market checks a couple of the boxes but comes up short in self-service and distribution. There are only a few products that cover all of these bases.

I'm happy to say that the power is out there, with the right tool and an open mind you can not only be a purveyor of the single source of the truth - you can augment it with multiple sources of context.

It took me a while to come to this, but it doesn't have to be either/or. "And" is almost always a better option.