Speed Trumps Perfection in Analytics

It's nearing lunch time and your mind is on the delicious burrito from the place around the corner. You're ready to chow down and take a break. Then you hear it.

The Ping.

Someone has sent you a message and when you open it up it's one of the key leaders in your organization and they need some quick information before a meeting. In 15 minutes. Immediately your heart races.

"Wait, I'm not sure about this!"

"Am I filtering this out correctly?"

"Does this chart look right?"

"That analysis will take more time!"

Relax.

When I worked in military intelligence, we often had to communicate extremely important information under extremely stressful conditions within extremely small amounts of time. It was always a challenge and you couldn't always pull all the data together and check it three or four or five times before the commander needed it.

That's why we used what's called a Probability Assessment Scale:

This scale was designed to allow us to make judgements of likelihood. It allowed us to pull together information under uncertain conditions and communicate that information. The chart approximates how judgements correlated with percentages, and then gives you the language to communicate those judgements and probabilities.

We can do the same in our analysis in the civilian world too.

Check out the scale I created above - it can be used to make judgements of confidence. Then, you correlate these judgements with percentages and now you have the ability to say things like:

"Our time to hire is likely around 20 days"

"We have roughly 200 terminations this quarter"

"Based on some quick math, our engagement score may be 50 for this quarter."

This kind of language isn't always ideal. But, being asked to give answers to complex questions in constrained circumstances isn't either and being able to communicate your level of confidence is always better than being unable to do so.

In analytics, speed is always going to trump perfection. Leave the statistical confidence levels for the lab and for when you have more time to verify. We're in the trenches and we need to understand our data enough that we can make judgement calls when needed.

Now, please don't misunderstand me. I don't want analysts running off and making all their judgements without quantitative data behind them.

But, I do want them to be able to say:

"This is my most likely time for hire"

"This is roughly our projected growth for next quarter"

"Based on the data we have available, here's the likely answer for that question"

This is far better than being asked for a number and giving a non-answer like:

"I'm not sure"

"I'll get back to you on that"

"That's a hard one"

As an analyst, the more you understand your data, the more confident you'll become in your judgements and the better and faster you'll be in handling tough questions from leaders. When you're in the due diligence phase - where you're really trying to understand your data and you're not sure what conclusions you can draw yet - you're going to have to rely on probability and not certainty. You need to have some way of communicating that to your stakeholders quickly, you need to be able to keep things moving.

What would be worse would be communicating things confidently all the time, but then realizing later you forgot to filter out one key category, or you accidentally truncated the data and now everything's shifted. You've made a judgement call, but by this time you're past due and it's too late. Things have been communicated, decisions have been made, and now you're left with a mess on your hands.

With that in mind, I hope this scale can give you a jumping off point for thinking about how you can handle these time-sensitive situations.

Don't expect to use this language with everyone, especially at first, and feel free to add additional explanation.But, as you get better at choosing your words and your confidence in your ideas increases, you should be able to move towards this more rapid and less-than-perfect language.

If you use this scale, please let me know how it goes. I'd love to hear your thoughts and how you implemented it in your organization.