Signal vs noise

Managing metrics in any organisation is a lot more complicated than it used to be.

At the start of my career, you were lucky to get weekly or monthly stats for most things. Now daily, real-time, and click-by-click updates are everywhere.

Trouble is, a lot of the additional data that has come our way in the intervening years is irrelevant, or at the very least unhelpful in the timeframe in which it's presented.

If a salesperson has a target of ￡10,000 a month, it's 9.30am on the first working day of the month, and your salesperson hasn't made a sale yet, does knowing that give you any clearer idea of how that month is likely to go for them than you had before?

Well, it depends on your business model, but almost certainly not.

Yet many businesses invest six figures every year to track information like that, even though it won't make the blindest bit of difference to the way you run your business. In no time, whether you like it or not, your inbox and notifications are being flooded with real-time updates.

Along the way, someone clearly thought this is a good idea, often at the prompting of a sharp-suited IT systems salesperson in my experience, but it rarely is.

Knowing that information is unlikely to make any difference to what the salesperson is going to do in the next half-hour, and it's unlikely to make any difference what the 17 other people who get the real-time stats are going to do either. That six figure investment was probably a monumental waste of money.

At least, at that point in the process...

Obviously if the same salesperson gets to the end of the month, it's half an hour away from close of business on the last working day, and they still haven't sold anything...now we have a different problem.

But even there...in that moment...is anyone who is privy to that information likely to do anything different between 4.30 and 5pm on the last working day of the month than they would have done anyway?

I doubt it.

Between those two extremes, is there a crossover point when it isn't too early to take action, while not being too late to do anything about a terrible month's sales results, if left unchecked?

A point at which some focused effort from the salesperson and/or their manager might turn this month's results around...or at least reduce the shortfall against their annual target which will be carried forward into next month?

Of course there is.

At that point, information becomes useful.

Before that, it's more of a distraction than anything else. It then remains useful for a while, before not being terribly helpful again because the salesperson is running out of time to turn this months' results around, almost no matter what they do.

Is it actionable?

In my days as a CFO, I refused to circulate information that wasn't actionable. I'm no fonder of bucketloads of junk I never have time to look at piling up in my inbox than anyone else, although I would concede my professional has a reputation for sending bucketloads of impenetrable reports around whenever possible. (For some people I've worked with, that reputation is entirely justified.)

There is a temptation for people who revel in data to circulate as much of that as they can find. But that's rarely helpful to anyone.

The costs of compiling data tables, charts, graphs, and whatever else constitutes an organisation's information flow is considerable. So employing people to produce reports nobody looks at - or even if they look at them, they can't take any meaningful action based on what they see - seems more than a little pointless.

I'd prioritised making reports actionable for a while before I heard the term "signal vs noise". I first learned about it in the context of the financial markets, but I later learned it pops up in other arenas too.

In financial markets, though, the "signal vs noise" question boils down to this: "can I make a decision to trade based on the information I've just been given".

If you can't, then it's noise...just the markets bouncing around randomly, doing what they always do.

If you can, it's a signal. Time to place that trade.

Sounds simple enough, I know. But most organisations favour circulating a torrent of data at every opportunity - and spending the GDP of a small country to do it - even though, most of the time, nobody is going to do anything different as a result of receiving that information.

When IT suppliers tell me they can give me real time updates of every metric imaginable, they think this is a major selling point.

For me it's a sign that nobody has thought clearly enough about what information they want from a system, so they've defaulted to sending everything to everyone as often as possible, no matter how irrelevant and non-actionable it may be.

Natural variation

One of the problems with expensively collected and distributed real time data is that it assumes every minute of every day is no different to any other minute of the day, when that's patently untrue.

It might be different if you're a huge multinational operating across multiple time zones, or you run an online ecommerce operation, but for most businesses there will be no sales activity between 5pm one day and 9am the following morning. For that period of time, the availability of real time information isn't a selling point.

Or during the day someone might take an hour off to visit the dentist, during which they don't make any sales. Yet you already knew they'd gone to the dentist so the real-time data system flashing red because Janet hasn't sold anything since 2pm is, at the very most, an unhelpful distraction.

Some days of the week might normally be stronger than others.

Some weeks of the month might be better or worse than other weeks.

And sometimes, people are just having a bad day. Tracking their minute by minute activity is almost certainly a waste of everyone's time.

If your end-to-end sales process involves making outbound calls to prospects with the objective of booking them in for a product demonstration and your average hit rate is 1 in 10, that doesn't mean every time you make 10 calls, one of them is a demonstration booking.

The laws of statistics mean that if you make enough calls for long enough, your hit rate will average out at 1 in 10. You might conceivably make 27 calls in a row without booking a demo, but calls 28, 29, and 30 all book a demo, averaging you out at your KPI of 1 in 10.

But statistics only works with large numbers, not small numbers.

So if that target is 10 calls an hour, you'll know in a day or two at most how your salesperson is performing.

But it that target is 10 calls a month, you might conceivably need to track performance for three or four months before you can take a view on their performance. Getting to the end of the month without a single demo booked means virtually nothing.

Much as providers of management information systems would like you to believe otherwise, a fairly high proportion of the torrent of real time information pouring off their systems is really not much use.

And if it's not much use, why are you collecting it, much less disseminating it around the business to distract people from the jobs they should be doing instead, which might put some more cash on your bottom line?

Your business model

Much of this goes back to your business model.

If you "hard sell" low-ticket items in volume, your data needs are very different from, say, a capital goods manufacturer who sells 30-40 industrial machines in the course of a year.

Your sales cycle is a good guide for what your information needs should be.

For selling low-ticket items in volume, a daily reporting cadence is likely to be about right, and you can use the laws of statistics to work out unusual patterns or salespeople who seem to be stuck in a long, inexorable slide in their daily numbers.

If you sell three or four high-ticket items a month, a monthly or quarterly reporting cadence is more likely to be appropriate. And the laws of statistics are unlikely to be much help because there just isn't enough data to make statistics work for you.

While some reporting is always helpful, that doesn't mean all reporting is helpful.

An area which is often missed are the leading indicators, which in turn are based on your business model.

For low-ticket, high volume sales it doesn't matter nearly so much because your sales cycle is likely to be registered in no more than a couple of minutes. You just sell as hard as you can and check your daily numbers.

But for high-ticket, less frequent, sales the actual sale itself is not all that relevant most of the time, because a sale is a rare occurrence under that business model.

However you are probably prospecting on a regular basis. You're having initial in-person meetings with clients. You spend a lot of time with them on diagnostics and systems specifications.

Even then, a real-time data tracker isn't going to tell you much here. But you might be surprised how accurate a forward sales forecast you can make just by knowing how many systems specifications meetings your salesperson has had this month.

If, on a fairly reliable basis, a systems specification meeting today turns into a delivery of some shiny new equipment to a customer three months from today, now you've got some useful information.

And while you still won't need real-time data tracking given that an in-person sales meeting in that depth is unlikely to happen more than a couple of times a day, the flow of systems specification meetings becomes much more helpful information than sales data.

Of course, your accounts team will report sales data in the monthly accounts, but the sales data will be "lumpy" and some months you might not have any sales at all.

But given that you will have some sort of conversion rate from systems spec through to actual sale, even if you do that at a pretty high 1-in-2 or 1-in-3 conversion rate, you now have twice or three times the information you had before about what future sale are likely to be.

Don't panic

Part of the problem with reports - especially now there's so much software around which can spit out graphs at the touch of a button - is that there's often a temptation to panic when performance isn't in line with some notional average or a monthly target figure.

That's usually when a whole raft of sub-optimal decisions get made.

When you see an average number or a target shown as a straight line on a chart, with an actual number plotted and a red "fill" between those lines if performance is below target and a green fill if it's above target, the temptation is to panic when you're in "red" territory.

While occasionally panic is an entirely appropriate reaction, most of the time, it's just that the timescale is off.

If you make three sales a month and haven't sold anything by lunchtime on the last working day of the month, don't discount hitting this month's sales target entirely, even though you've spent most of the month so far in "red ink" territory.

And, provided you've been tracking the leading indicators appropriately, even if you end the month with zero sales, it's highly likely that the three sales from "last month" will come in during the first couple of working days of the following month, and you'll still have a whole month to make three more sales before 5pm on the 31st rolls around to keep your long-term run rate intact.

If you try to manage a long sales-cycle business on short-cycle data flows, you'll almost certainly make bad decisions - you just need to hold your nerve sometimes.

Equally if you try to manage a short sales-cycle business using just long sales-cycle data, things can easily have gone completely off the rails before you know anything about it. That's not a good outcome either.

So when it comes to data and reporting, it's worth checking two things:

1- Does the reporting cadence make sense in the context of your sales cycle

2- Is the information being circulated actionable?

Anything over and above that is likely to be unhelpful at best, not to mention expensive to compile and distribute.

When a business is going through hard times, there's a temptation to think that you need more data, more information, more reports, so you can work out what's going on.

Often the answer is that you need less data, less information, and fewer reports.

When there is too much "noise" in your reporting - either because of a timescale mismatch or because you're sending round a never-ending torrent of mostly irrelevant, non-actionable information - then odds are you'll miss most of the "signals".

It's easy to develop some sort of number-blindness and miss most of the key insights you need to take action in your business.

Next time a report turns up in your inbox, ask yourself whether it's helpful in the context of your business cycle and whether it's actionable.

If it's neither of those things, seriously consider switching that report off.

At the very least you'll be giving people time to manage the activities in your business that really matter, instead of spending their time battling against a never-ending fire hose of mostly irrelevant data.

You never know. They might spend their time doing something more productive instead.

Just because you can, doesn't mean you should. That's good advice for constructing company reporting systems, and it's also good advice for using AI to write articles for you. No AI was used in compiling this article, so if you don't like it, that's entirely on me.

But if you do like it, feel free to give it a like and a share. I'll certainly appreciate it.