Watch Out for the "Big Data" Steamroller!

Forget statistics for a moment: Responsibly "parent" your big data to make it "mature" first

Statistics: the art and science of collecting and analyzing data.

People think that if you collect enormous amounts of data you are bound to get the right answer. You are not bound to get the right answer unless you are enormously smart. Bradley Efron

There has been an explosion in new technology for acquiring, storing, and processing data. The “big data” movement (and its resulting sub-industry, data mining) is becoming more prevalent and having major effects on how quality professionals and statisticians?– and everyone else?–?do their jobs.

Big data are a collection of data sets that are too?large and complex to be processed using traditional database and data-processing tools. Any change this big will require new thinking.?

Rocco Perla, a colleague for whom I have the utmost respect, feels that even though there is now unprecedented attention and focus on analytics and data-driven decision making, it has also introduced a number of challenges.

It is poor practice to rely on whatever data happen to be available or to assume sophisticated analytics can overcome poor data quality.?The fact that data reside in electronic files says nothing regarding the quality of the data.?Observational data are teeming with reproducibility issues, especially if it's a resulting merge of many different sources.??

Intuitively, as the amount of information increases one would think that the degree of confusion decreases.?But this is true only to a point because the situation eventually reverses, reaching a point where more information leads to increasing confusion – low amounts of information and high amounts of information will both lead to a high degree of confusion!?There are real economic implications of information overload – including a loss of up to 25% of the working day for most knowledge workers.

W. Edwards Deming often made the point that information is not knowledge.?The speed of today’s worldwide instant communication does not help anyone to understand the future and the obligations of management.?Do we really need constant updating to cope with the rapidly changing future??It could hardly be accomplished by watching every moment of television or reading every newspaper!

Data Maturity = Data Sanity

Perla introduces a concept he calls “data maturity” to begin to make sense of all the available data, especially with the growing sub-industry of project work inherent in many improvement philosophies.?It has five characteristics:

1. Projects and their supporting data (eg, reports, dashboards) are viewed as a resource expenditure that adds various costs and complexity for needed support, which should be only temporary.

No more automatic “data on demand.”

• Organizations that are not data mature often view requests for data as going into a magical black box that, with limited input, thought, and resources, is able to produce the desired end product perpetually.

2. Projects do not go on forever.?Any project needs formal closure to be retired, after which its measures are also retired.

Any transition from diagnostic and testing data to the consideration of collecting data to hold any gains should be done deliberately and with thought.

• It is not unusual in many organizations for data, reports, and projects to reach a point where they are never or rarely looked at by the requester in the extended future.
• If one is not careful, new measures become additive to old measures and are allowed to continue in perpetuity and to clutter the environment.

3. All measures are operationally defined.

In data mature organizations, operational definitions are viewed as sacred, because they are the only way to ensure a shared understanding of the work to be done.

• It is not splitting hairs to formally define adjectives such as “good,” “reliable,” “unsafe,” or “ unemployed” so that, regardless of the person measuring the situation, they would either come up with the same literal number or decision (eg, thresholds where an environment goes from “safe” (x=0) to “unsafe” (x=1)).?
• Even though people may not agree on the actual definition, they will agree to use it and continue to use it as long as the ability to make the desired decision inherent in the objective of the collection remains strong.?The objective here is to reduce the “human variation in perception” factor, which will contaminate any data and render them virtually useless.
• Management cannot knowingly allow for idiosyncratic or capricious interpretation of a measure, especially if there is potential for distortion to meet an arbitrary goal.?

For example, is Pluto a planet (x=1) or isn’t it (x=0)??It depends:?What is the objective for asking this question??(Never mind how you feel about it!)

4. Improvement measures are clearly and explicitly linked to any changes being tested in the system – and they are collected frequently over a period of time.

The purpose of improvement is to understand?–?as quickly as possible?–?if the changes made to a system are leading to improvement and to inform the next test.

• Many improvement practitioners shy away from annotated run and control charts because direct causal links between changes and outcomes often are not possible.?

5. The dominant form of analysis includes charts of data collected over time to determine, and react appropriately to, common and special causes of variation.

Organizations that are data mature understand the importance of examining data over time and using these charts as developed by Walter Shewhart (mainly control chart for individuals)?to distinguish between special and common cause variation.?

• These charts are based in what are called analytic statistics and neither resemble what is taught in most academic courses (enumerative statistics) nor involve formal hypothesis testing.??

How does one filter this plethora of information and make sense of it all??

In my experience, virtually without exception, these ideas begin to seriously challenge the way leaders and organizations currently think about information, data, and decision making.??Environmental pressures remain constant, and technology is not going to slow down.?This creates more dependence on the tremendous inertia of the status quo and a toxic blindness to the deceptive power of this different approach's counter-intuitive simplicity.?This blindnesss especially makes people prone to...

...the most seductive waste of all?

Hand-in-hand with the explosion of big data will be an industry trying to sell you solutions for analyzing it.?In fact, a client of mine sent me the figure below:

In this case, the vendor promises highly visual, easy to understand information that will allow insight into not only what the level of employee engagement is, but also the level of leadership effectiveness within each department.?They claim their analyses are statistically robust and repeatable over many years (and I'm willing to bet that, unfortunately, most of them use this all too common (wrong) analysis). Note how they "delight their customers" by also throwing in red, yellow, and green color-coding.

Another claim of theirs that is some possible good news, but for an entirely different reason:?this one tool is a fraction of the cost of a typical survey.?

Well, at least you could save money on all those ongoing silly customer satisfaction surveys?–?how many of your electronic data files are filled with those?

Data has its place, but the challenge is to maximize its ability to serve us humans with all our limitations?–?not the reverse.

Perla concludes that the most effective future leaders will leverage this approach to data with a vision, energy, intellect, and moral compass that come from within?–?not so much from a report or balanced scorecard.?

Let's stop the "data tail" from wagging the dog, shall we?

====================================================

Chapter 2 of my book Data Sanity?demonstrates Perla's point #5 of the deceptive, awesome power of plotting data over time for solving problems of everyday leadership (10 examples of real data from everyday work).

Chapter 5 explains a "mature" data philosophy and Chapter 7 teaches statistical stratification using p-charts and u-charts to compare performances.

Regardless of your improvement approach, Chapters 1 to 4 of my book Data Sanity teach a robust, results-oriented leadership philosophy designed to catalyze transformation to a "built-in" culture of excellence.

See my LinkedIn profile for more information (downloads of Introduction/Preface and brief chapter summaries. It's not only for healthcare).

Read Rip Stauffer's review, published in Quality Digest

• Easily ordered through Amazon
• An ebook version is available through the publisher for all e-readers,?including iBook, Nook and Kindle (purchase price also includes downloadable .pdf)
• For UK readers who want a hard copy, it is available?on Amazon UK