The Devil Is In The Data - The Endless Hunt For Data

In 1939 Metro-Goldwyn-Mayer released The Wizard of Oz. Towards the end of the film the protagonists finally arrive at their desired location, in order to meet the great Wizard of Oz. Confronted with the ominous presence of the great wizard, Toto the tiny dog casually makes his way over to a concealed curtain in the corner. Next, he slowly pulls aside the curtain, revealing an old erratic man frantically pulling levers, and twisting dials whilst yelling into a microphone. And thus, the Great Wizard of Oz was revealed in his true form, warts and all.

This is a great analogy for the emergence of data in the corporate and commercial worlds. In 2006 Clive Humby coined the phrase ‘data is the new oil,’ and the growth of Google, Amazon, and Facebook (to name a few) are all testament to the lures associated with this largely misunderstood resource.

The temptation for many organisations is to ‘collect data’ and present it on sexy dashboards with various graphs, presentations, and tables. This is often done in response to:

• First sight of a competitor’s capabilities and the urge to create a similar capability because it must be what is serving them well. 
• The need to look busy or proactive
• The desire to feel informed in order to better understand
• Supporting decision-making

The creation of these tools is expensive, distracting, and without focus and prioritisation can be dangerous rabbit holes in which to get lost in.

In 1978 The Hitchhikers Guide to Galaxy was released by Douglas Adams, originally presented as a radio comedy broadcast on BBC4. A comical scene exists whereby an alien race develops a supercomputer called ‘Deep-thought’ in order to answer the ambiguous questions to ‘the great question – of life the universe and everything’. After 7.5 million years of processing time, it comes back with the disappointing answer of ’Forty-Two’.

Deep-thought goes on to explain,

“I think that the problem is that you have never really known what the question is…You have to know what the question actually is, in order to know what the answer means”

So herein lies the problem. So many organisations are frantically collecting as much data as they can without any coherent understanding as to why, or for what purpose.

THE IMPORTANCE OF STRATEGY

Strategy should underpin everything that we do with our teams. The link between operational efforts and our strategy direction should be strong, coherent, and measurable.

As a minimum our organisational strategies should include:

• Vision – Where we are heading
• Mission – Why we do what we do and how do we know when we have got there.
• Scope or Services – How we do it
• Objectives & goals – The nuts and bolts of how we will grow our influence, value to others, and our teams. Also, how we intend to measure it.

Our organisational strategies need to deeply influence our decision-making. Data serves the purpose of reinforcing decision making by:

• Creating faster decision loops
• Distinguishing between different courses of action
• Disproving assumptions and turning them into facts
• Measuring the success and validity of our objectives and goals
• Sensing where emerging opportunities are presenting themselves
• Linking with our understanding of risks and opportunities

But in order to achieve all this, data must be used as a surgical weapon, not a blunderbuss/shotgun approach. Failure to do so will only cause more problems than you had originally.

POOR DATA COLLECTION

Data is not the silver bullet people often choose to rest their projects and careers on. Any statistician worth their salt will describe how information can be manipulated to tell the desired story. Moreover poor or lazy collection of information will lead to terribly unreliable outcomes. For example:

1. Size of the pool. If the pool of information is too small, it will not provide an accurate depiction in which to make reliable deductions as it is not representative of the demographics or groups you are seeking information from.
2. Length of the analysis. If you are capturing information from a very small period of time, then it is likely affected by environment and external influences which will lead to a false reading. If you ever need an example of this, refer to those people that follow the stock market every few minutes, but have committed to a longer-term strategy. The fluctuations do not tell an accurate story of the growth or decline of a particular group of shares.
3. Scope of the study. When the scope of the study, or the question you are trying to answer is poorly defined, you are likely to collect data that doesn’t serve a purpose. Refer to ‘Depp-Thought.’
4. Methods of collection. There are countless different mechanisms of collecting data and then analysing it. If the wrong method is chosen, then it will significantly distort the results. There are professions that specialise in data collection for a reason. They use different tools to achieve different things!
5. Reliability of the sources. If the information is pooled from dubious sources then the validity of the findings will be questioned later. It is very much a case of ‘crap in, crap out.’

ASKING THE RIGHT QUESTIONS

Before diving into data-collection we must know why we are committing to it.

The questions must be geared towards answering an overarching concern or opportunity and should directly link to our project scope or organisational strategy.

The questions must be geared towards refining and tightening the scope of the collection in order to narrow the ambiguity of the project.

To get started you could ask:

• What information do I need to collect, in order to answer what question?
• What decisions need to be made? What information is required to help make that decision?
• What information do we not need to collect? What don’t we need to collect?
• How much information do we need in order to answer the question? When can we stop and make the decision?
• Does this information perform an important function, or is just creating white noise? (hint: link to decision making).
• Can I further tighten the filters and variables in order to target a specific question?

HUMAN BEHAVIOUR

The list of human shortfalls that affect data are too lengthy to mention. But here are two very important ones that are often overlooked, bias and subjectivity...

Bias

Bias comes in all different forms including everything from ‘confirmation bias’ where we actively look for information that confirms our original hypothesis or stance, through to ‘selection bias’ when data is selected subjectively, and everything in between.

Needless to say, that our own personal biases completely undermine our ‘objectivity’ if not cross referenced against other sources or mechanisms.

Examples of this occurring include (source: Cmotions):

·      Poorly articulated questions in questionnaires

·      Choosing people from a demographic that will support our claim or stance

·      Breaking people into poorly defined or irrelevant groupings

·      Measuring things incorrectly

·      Non-random selections

Subjectivity

Everyone is experiencing the world in very different ways. Moreover, the way we feel at a certain time can have significant implications on the way that we collect information, engage with participants, and interpret information.

If you want accurate and useful data you need:

·      A plan (the right tool for the right job)

·      Structure (sequencing and staging)

·      Objectivity (multiple sources of data, and quality checking collector’s activities)

·      Quality assurance

·      Professionalism and Discipline (stick the plan and don’t jump to assumptions)

If you do not have these things as a minimum, your efforts are likely in vain, and you are likely wasting everyone’s time.

CONCLUSION

Data is a new fad.

It sounds cool, but most of the time it doesn’t mean anything or serve any purpose because we don't give it the respect or attention it deserves.

If you are collecting data without knowing why. Stop. Reassess. Fix it.

I would suggest that if you are looking for data that prove to yourself that you are doing great work then you are likely using it for the wrong reason.

If you enjoyed this article and would like to join us in other thought-provoking pieces, please join us at our blog.