The life of pies

I'm always amused - and more than a little disconcerted - by people who claim to "manage by the numbers".

Numbers without context mean very little and they're often used by people who want you to believe what they believe - sometimes knowingly, like a politician drumming up support for their political views, sometimes unknowingly, when people just don't understand the full picture.

I've never yet met an underperforming salesperson who didn't have a raft of statistics to show how hard they were working and how their lack of results was always someone else's fault. High-performing salespeople working in the same business, selling the same products, just hand in a note of their sales numbers at the end of the month and miraculously reach or exceed target while only needing a single number.

It's the same for operations people, engineers, marketeers, accountants, HR people and any other profession you can think of.

Numbers are helpful - essential even - to track progress towards achieving the results the business needs. But any time someone presents a set of numbers to you, remember the numbers they give you are intended to serve one single purpose - to convince you that the person presenting the numbers is doing a great job.

They are highly likely to be "the truth" insofar as they go. I very rarely find people just making stuff up. But they are equally highly unlikely to also be "the whole truth and nothing but the truth".

That's why context matters. An example might help.

Pies

If we've met in real life, it probably wouldn't surprise you to learn that I like a pie. Sweet or savoury - either is fine.

But let's take this simple foodstuff and use it to illustrate the importance of context.

The last pie I bought needed to be heated in an oven for 20 minutes at 200C. And it was very nice, to be fair.

But what if I deep-fried it hot oil for 20 minutes at 200C? It's the same temperature, and the same amount of time, right? Well, I'm not sure that would be anything like as nice to eat.

I could also pop it in a frying pan and use a temperature probe to make sure the insides were heated to 200C for 20 minutes, or do the same on a grill.

Perhaps I could heat it for 40 minutes at 100C - after all, exactly the same amount of heat gets into my pie, doesn't it? Half the heat for twice the time sounds like the same thing to me.

Equally I could microwave it on an equivalent setting for 20 minutes, or stick it in a wok.

There are probably lots of other ways I could heat up a pie for 20 minutes at 200C, but without the context of knowing that it needs to go in an oven, and understanding that no other cooking method will produce a similarly tasty result, just knowing that the pie needs cooking for 200C for 20 minutes isn't much help.

And that's before we decide whether we're serving the results with mashed potatoes and peas or ice cream.

That's why numbers alone, shorn of their context, are of remarkably little use in making good decisions.

And if you're only getting a partial picture of the numbers - because the person presenting them is only sharing the good stuff in the first place - you end up making decisions with only a tiny proportion of the information you'd ideally like.

Painting by numbers

Part of the problem is that a lot of business education and training in recent years has focused around running businesses like a painting-by-numbers kit. It's all about extracting and manipulating numbers, loading up spreadsheets, cranking up Power BI, and making pretty charts.

There's a role for that, of course, but mostly charts and graphs strip away nuance, they don't add to it. Thousands of data points are turned into a single line on a graph, but that line is probably the least useful of any piece of management information: "the average".

Knowing just the average of any data set is of very little help, and it can be dangerous because so many sub-optimal decisions are made on the back of that single number.

Consider this by way of context.

Let's say the average of some performance metric in your business was 10.

In lots of organisations, that's all they would use for budgets, financial plans, targets for the workforce, HR-led competency reviews, and so much more.

So ask yourself - would you do any of those things in the same way with this context added:

While the average is indeed 10, it's made up of six separate people's results. Their results are as follows:

0, 0, 0, 20, 20, 20

With that context, knowing the average is 10 is all-but meaningless. And it could be meaningless on both the high-side and the low-side.

What it I told you that the "zeros" had only started work today, and the "20s" had each worked for the business for 5 years plus?

In that scenario, you'd expect an average far higher than 10 in a few months' time, so that would not be a sensible figure to base any future budgets and financial plans on.

Equally, what if I told you that two of the "20s" had just handed their notice in, but that everyone else had been with the business for at least a year? That average of 10 isn't looking much like a solid basis for a forward financial plan to me.

Rather, it looks like you got lucky with a handful of good performers and probably don't really understand what's going on, or you wouldn't have the three zeros.

But what about if the three "zeros" were the boss's children...would that context make a difference to how you interpret the numbers above?

Or, indeed, if the "20s" were the owner and their two siblings who had worked in the business for over 30 years each?

The perils of science

I know it's tempting to look at a set of numbers very simplistically and make the "obvious" decision. But that's nearly always wrong.

In my experience, it's especially likely to be wrong if numbers are compiled and reported to several decimal places. And that goes double if someone with a science or engineering background has prepared the numbers.

In those disciplines precision to the nth degree is always the objective - and rightly so under laboratory conditions.

But in a business setting, a lack of precision two or three decimal places in isn't usually the problem - being more precise doesn't bring any more of the context you need. It just gives more depth to the context you already have.

Knowing, for example, that one of our "20s" above actually scored 19.895 and one of the others scored 20.105 makes absolutely no difference to how we'd interpret the dataset above. (Or at least how I'd interpret it - hopefully by the end of this article you'll see things the same way, if you don't already.)

The big question really is "why do we have one group clustered around zero and another group of equal size clustered around 20?". More precision in the measurement of each individual's results won't help solve that problem.

People who are used to working under laboratory conditions are especially susceptible to this thinking blind-spot because they often forget that laboratories are closed systems where pretty much every variable is controllable and extra precision can bring you greater insight.

Whereas anything any business does is, in reality, having to sit in a primordial soup of endless, seething humanity, the vast majority of whom don't know and don't care about the intricacies of the science. They just want your business to do whatever it's supposed to do for them.

In a lab, everything is rational. In the real world, almost everything is irrational.

That's why context matters. Not because it means you'll have a guaranteed answer that you'll be able to trot out in a mechanistic fashion across your whole business.

But because you'll get an insight, a perspective, a sense of likely intention from that context which will help you make better decisions using the (necessarily imperfect) information you have available.

Size doesn't matter as much as people think

And that's true no matter how much data you have, how frequently you analyse it, how ferociously you chase down every umpteenth decimal place of results available through your company's management information systems.

Sometimes the context-shift you need is to shift up a couple of levels from the "right at the coalface" level of data to the "helicopter view" level.

I can still remember a conversation with a salesperson who reported to me in a previous role.

He spent the best part of an hour in his performance review telling me how brilliant he was at every aspect of his job. He had stats, charts, and graphs to prove every point - every one of which was absolutely true in the very narrow context of the information presented.

On a "coalface level" he was right. At the helicopter view level he was miles behind his sales target.

The real problem here - the context if you like - was nothing to do with his activity level or anything in his stats. It was all about his mindset. That was the problem we had to solve.

Because things weren't going well, he'd end up talking to any remotely potential customer who gave him the time of day. He did a poor job of qualifying his leads and wasted enormous amounts of time pursuing potential customers who were unlikely to give him any business - and almost certainly not at the sort of prices we charged.

And that's why, despite being an accountant by profession, I'm not that obsessed with numbers. Numbers alone rarely give you the context you need to know what's really going on.

Sure, they might tell you there's a problem to solve. In the example earlier, knowing that, out of six people doing the same job, three get a 20 and three got a zero tells you there's likely to be a problem to solve. It doesn't tell you what the problem is or how to solve it.

In fact, in one of the scenarios above, I'd argue we didn't even have a problem. If the three "zeros" had only started work today, I wouldn't have a difficult agreeing that their performance was highly likely to trend towards the 20-mark over a period of time.

I might want to have some idea of the timescales, and I might want to track incremental improvements towards that point, but I'd argue there isn't a problem at all here. We just have a situation that will solve itself over a period of time, assuming the right coaching and support is in place for them.

False positives for a problem

One of the biggest dangers of manging by numbers alone, especially if you have lots of data you use this way, is the way it can generate false positives (and false negatives) for the unwary.

In other words, the numbers can make it look like you've got a problem even when you haven't, and vice versa.

Again you need context, but taking information at face value can easily lead people up the garden path.

Some years ago, I worked in a highly-regulated sector where it was normal to have league tables that collated performance across the industry and ranked each organisation against the sector as a whole, in which there were 200 or so organisations.

One year, the division I ran got an 82% (vs a >80% target) and showed an improvement on the previous year's league table position.

The following year, we got 83% on the same metric, but dropped a few places in the league table because the rest of the industry had improved. To listen to my boss, it was like the end of the world.

The number of working groups, PowerPoint slides, strategy away days and goodness knows what else this spawned doesn't bear thinking about, much less the cost incurred and time taken to endlessly strategise on the subject.

The fact we had improved year-on-year was barely noted due to an obsession about a single datapoint - the industry league table position.

And this was the case despite the fact that there was no way of knowing what any other organisation in the sector was doing until 12 months or so after each annual reporting period as that's when all the industry stats were collated and published.

The year after, we got 83% again, but shot up the table by about 20 places. While my boss had castigated me over a modest drop the previous year, and liked to boast about the league table position to his boss and external stakeholders, he never once congratulated me on that massive achievement.

Not that I was expecting it, particularly, because I was disappointed that, despite all our hard work in the meantime, we had another year at 83%. But if a drop in a league table position is a bad thing, surely shooting up the league table is a great achievement?

Sounds improbable, but it's true...

However the real context you need to know here is this:

1. The methodology by which this league table was compiled was fairly dodgy, and sampled entirely different groups of people year-on-year making any consistency in meeting expectations impossible and also leading to highly volatile results (within a range) as there was no consistency between last year and this year.
2. The results were all very tightly clustered around the same level. Pretty much everyone achieved "about 80%" on this metric. A few outliers apart, almost everyone was somewhere between 75% and 85%.
3. However, by the time it was taken out to two decimal places, someone on 80.03% would be higher up the league table than someone on 80.02%, even though I'd still defy a single customer to tell the difference. So tiny fractions of a percentage point increase, or decrease, in absolute terms would have you either rocketing up the league table or plummeting down it.

This led to a "feast or famine" situation, where a jump up the league tables led to champagne corks popping in the CEO's office and drops down them resulted in a deeply funereal air at the next senior management team meeting.

The underlying reality - actually the only thing real customers cared about - was entirely capable of getting better in "down years" and getting worse in "up years". The randomness baked into the methodology made this metric wildly unpredictable and almost impossible to interpret, so it was a pretty terrible metric all ways round.

But the bigger point here is that this was a metric which regularly presented both false positives and false negatives. It was easy to fool people into thinking they had a problem, when they didn't. And into thinking they didn't have a problem when they did.

If you just looked at the league table position and "managed by the numbers" you'd form one conclusion.

If you understood the context, you'd spend most of your time wondering why a methodology which was almost guaranteed to produce skittish, inconsistent results from one year to the next was used to manage anything at all, much less to treat it as one of the be-alls and end-alls for organisations working in this sector.

The context for context

That's why the context matters when you're dealing with numbers.

Whether you're cooking pies, managing underperforming salespeople, or comparing sector-wide industry statistics from one year to another, numbers on their own mean very little.

Sometimes they at least let you know you've got a problem, but even that isn't infallible. False positives and false negatives abound.

As do what I call disconnects, where a movement in metric A doesn't necessarily mean the end result will be any different to what it is now. No matter how much you obsess about metric A, it might not move the needle in respect of the end-result you're trying to achieve.

Add in some context, though, and the way forward becomes a lot clearer.

Not necessarily as clear as a freshly polished piece of glass, but those situations are rare in a business setting anyway. But clear enough.

You can see what you need to see well enough to make a decision and be fairly confident that your decision will have the desired effect on the end result.

So don't just "manage by the numbers", despite what the last training course you want on said, and the sounds of "what gets measured gets managed" ringing in your ears from all the times you've heard someone say that.

Often, managing purely by the numbers is little better than guessing.

Add some context first, then make your decision. It's likely to be a vastly better decision if you do.