What's in the numbers

Leaders often tell me that they desire a culture that uses data to make decisions. When I probe further, the underlying desire is for better results through objective decision making. Getting to this is not as simple as collecting measurements. The challenges are often more behavioral than mechanical such as a propensity to create solutions with limited information, a tendency to make bias interpretation of the data, or a poor understanding of the limitations of measurement. To achieve the end goal of better results all of these challenges must be addressed. 

In companies where anecdotes, strong personalities, and politics rule, data is not of much use. In these companies I find leaders over react to issues based on “who” is upset versus the true size and impact of the problem. This causes organizational waste in the form of missed opportunities and misaligned priorities. On the path to objective decisions this is the first level to get through. Leadership must become aware that there are behaviors at the source of the problem that data will not directly correct. Addressing these behaviors and establishing values that reinforce objective decision making will make data-based decisions possible and effective.

Good measurement makes performance transparent, helps people outside the process see cause and effect relationships, and aids leadership teams in gauging priority. It is important to note the word “good”. The data is not always good. Data is the result of measurement. Measurement is a process and as with all process it has waste and variation. In fact, data is never perfect and often all we can do is make it “good enough” for the decision we need to make. This is the next problem for leaders to solve. Unfortunately, I find that when organizations get mired in this problem, they tend to revert to those first-level issues. To prevent this, there are both tangible and intangible issues to consider. The tangible issues are well known and resolved through better tools and methods. The intangible issues need consideration as well.

When I am teaching Lean Six Sigma, I tell the class that for some Six Sigma held the promise of providing a mathematical equation that could answer any question or “automate” a decision. It is the panacea of leadership to make flawless decisions.  If it were only possible to measure all the variables a leader must consider, place them into a formula, then poof, display the perfect decision. To this end, in my Black Belt education, I learned the concept of the “Transfer Function.” It relates important causes to measurable effects. It allows us to look at each cause as a measurable variable that is mathematically modeled to determine the effect, with error of course. So cool, the concept is fantastic, but it can be over extended. As any good statistician will tell you, “all models are wrong, but some are useful.” It is a folly to believe that we can simply remove judgement and trust an equation. It does not work perfectly in practice and as a result, leaders decide not to trust any numbers and stick to their time tested “gut instinct” or “shoot from the hip” decision making. Overcoming this intangible component of using data for better decision making requires a balanced approach.

Data is a guide. It won’t give you an answer. It can effectively challenge assumptions or confirm solutions. If the underlying goal is for better results, I suggest starting with data and then asking how it can be wrong or misleading. How can the data be inconsistent in telling you reality (precision) or leaving an important consideration out (accuracy)? In that space of the unknown is where a leader’s judgement is critical and true leadership occurs. A good example is that I will ask leaders how they measure the productivity difference between an inspired work force versus one with low morale. They can’t quantify the answer, but they know there is a significant difference. A measure of productivity is an important starting point, then we need to use judgement to determine if we have effectively inspired people.

There is an adage that I like:

 “Not everything that can be counted counts, and not everything that counts can be counted.”

-William Bruce Cameron

If the underlying goal is better results through objective decisions, then data should be considered the starting point. To do this effectively, leaders must create an environment that desires data, desires truthfulness in data and understands the reality that it will never tell us the whole story. Leadership and judgement will always be required.

If you find this topic interesting I recommend you listen to:

Jerry Muller on the Tyranny of Metrics. You can view it here.