When good metrics are bad
For all the value they bring, business metrics can also lead to trouble. There are the people who rely too much on metrics (and ignore other context). Then you have the poor choices of metrics (which measure the wrong thing). Or the ill-fitting presentation thereof (like that fancy dashboard that no one looks at any more).
Something you rarely hear about, though, is when a good metric is bad.
Yesterday Solomon Kahn kicked off a thread on metrics and it reminded me of an underappreciated aspect thereof: simplicity.
If you work alongside data scientists (or quants, or similar roles) you probably hear the phrase reduction of dimensionality a fair amount.?This is a nice way of saying "we're going to take a large amount of info, then boil it down to a smaller set of info, so that it's easier to digest."?(This usually improves ML modeling, so that's a Good Thing.)
A metric is also a form of dimensionality reduction.
Credit scores??A project's red/yellow/green status? Business valuations??All of them squish several months' or years' worth of fine-grained records into something simple.
And this is fine, so long as you remind yourself that a metric is -- by definition -- an oversimplification.
It becomes a problem when you forget how much detail you lose on the journey to that simple number or status.
The take-away: It's time to take a look at your metrics. Which ones do you actually use? When did you last inspect them, to reminder yourself how they came to be, and what messy details they hide?
Finding insights in surveys
2 年Can’t agree more. So often, I hear “Be data driven”, “collect data to support your decision”, but often metrics are collected and used beyond their purpose. It’s important to regularly review why you are using a metric as well as what the metric is telling you, discarding if it no longer makes sense. I like to think of metrics as ice cream. They are tasty when you first get them, but if you hold on to too long it melts and all you have left is a mess on your hands. ??