'When' you measure is just as important as 'what'

I've recently come across the debates that Peter Drucker never uttered these words.

Given he has uttered so many other useful statements, I'm not too fussed if this really isn't him, as I don't intend to bend the quote too far. Here's what I'm considering: in its most simplest and practical sense, the very act of putting thought to measuring something, invariably will change its outcome. We know this to be the observers paradox, and it can cause doubt around the veracity of what we measure.

I think of it similarly to how the first calorie-counting diet you embark on invariably results in weight loss - just by virtue of measuring what you eat. This weight loss occurs despite the fact that you know that the diet didn't really have any magic (or science) behind it, and as soon as the counter stops clicking, the gains stop.

The same seems to occur when you implement a new piece of technology. Whether its a new intranet, a customer experience platform, or something else - measuring 'success' afterwards always seems to have these giant gains, without any of the associated business benefits. Why is that?

But how do we know it is actually working?

Picture this: you’ve invested significantly in researching, procuring and deploying a CRM for your service desk. You’ve got yourself something shiny and new, and you’re excited about it. Results from the first few months post-deployment are encouraging - customer support enquiries are all automatically routed to the correct department and the team are reporting that they are getting closed "super quick".

At the next board meeting, you proudly share these stats around first touch resolution. In response, you get asked, ‘but how do we know it is actually working?’.

It is only really working, if the bottom line changes. Things like first touch resolution, customer satisfaction, NPS, are only leading indicators for customer retention rates and hence, annually recurring revenue. But unless someone is thinking about that at the start, it is easy to get distracted.

This scenario is more common than you think. A lot of people implement a CRM, or a marketing platform, or something shiny, and they might go through a good project process - but they don’t think about what it takes to align the work with the original business problem.

Leading indicators are not bad. They just can't be the end game. Businesses are often preoccupied with these metrics because you can immediately see a strong result, and not have to be held accountable to the expected financial outcome with this. Even more problematic is that often the windows that these leading indicators are evaluated across have no real business meaning. Why is it valuable that your mobile app increased its engagement by 25% during a particular 14 day measurement period?

So how should you measure?

Think about the original business case that justified the investment. Without trying to sound too cynical, when you boil it all down, it would have been oriented around increasing revenue, decreasing cost, managing risk, or consolidating power - in some shape or form. That's okay - be clear about it, and write it down, and be clear about the unit of measurement. Revenue is simple enough, but risks should be weighted and prioritised so that they can be measured.

Now that you know your ultimate measure, define your measurement period. You need to know when you expect to see change in that measure, or under what conditions that occurs. There's no sense saying you want to increase average customer value and plan to measure it daily, if your campaign is a multi-touch program over a number of months.

Once you have clarity about your real measure, then look at the leading indicators - pick ones that you know have shorter measurement periods, so that they really can be ... leading. The goal here is to accept that if your ultimate measure only gets measured at x cadence, you need leading indicators that will give you the heads up before you reach that x period of time. Otherwise, it is meaningless.

I'm already mid-way in my implementation. When should I start doing this?

Now. Now is when you need to start this.

If you now have a clear idea about how you'll measure success in the future, you need to measure a baseline, and there's no day like today. If you’re not live yet, you have an opportunity to establish your line in the sand. If you start tracking something, even if it’s manual, you’ll have a data point that can be used as a future point of comparison.

But even if your platform is already up and running, it is never too late to begin monitoring user behaviour.

You have to start somewhere.

Appoint someone to own your metrics.

This might all sound like pretty basic stuff, but it is all too often that these issues are being dealt with as afterthoughts or, shockingly, not at all.

Yes, it’s a lot of work. In fact, for organisations with larger implementations, it can even be a full time job. And why shouldn't it be? You worked hard to build a business case to deploy these tools - there was a tangible business outcome expected. Why shouldn't that be someone's job to see that through?

The first step to harnessing the true power of your CRM / DXP / MA / Digital Workplace / insert-your-shiny-investment-here is to make someone accountable for the ongoing success and performance of it. They need to be constantly looking at what people are doing, or not doing, asking the questions that need to be asked and then tracing it back to whether there’s a usability issue somewhere in the solution. Or, maybe it comes back to not enough training being provided.

You can’t successfully push forward unless you’ve thought about some of these topics beforehand. Making it somebody’s job just to think about it is the starting point.

If you’re currently implementing or have just deployed some new tools, you’ve taken the first steps toward that bright future your business is striving for. Now it’s time solidify your investment and appoint someone so you can realise its full power.

To summarise:

1. The very act of measuring something brings about improvement.
2. Improvements can be a distraction - unless they are directly connected to business outcomes which are measurable over agreed periods of time.
3. Use a combination of tangible business outcomes and leading indicators to build a measurement plan which allows you to evaluate the real commercial success of an investment and ensure it meets the original business case.
4. Don't wait to begin this - do it now.
5. Make someone accountable for all of this.

Thanks for reading this article. It was originally inspired by this post on the Squiz website, but has been adapted to broader use cases - since it really does apply to any technology investment.

If you liked it, you may appreciate some of my other posts:

• How to ensure your tech project doesn't lose its fizz
• 5 ways to get more out of your CRM investment
• Top tips for technology adoption (includes bonus picture of cute puppy)

Please feel free to use the comments box to say you love it, hate it, or have better ideas. Or if you just want to share your own experiences of #FakeStats, feel free to add too.