I’m Busting the Myths About Innovation Metrics

Companies want and need to innovate, but most use the wrong innovation metrics and lack effective ways to track their performance. Determining which metrics to use and how to use them remains one of the poorest understood aspects of true, breakthrough innovation.

So, I want to shed some light on the secret to successful innovation metrics and help you start to create better results for yourself and your customers.

First, let’s clear up some myths and misinformation about tracking success that may be hurting your efforts.

Everything You’ve Been Told About Innovation Metrics is Wrong

It’s no wonder companies are missing the crucial capability of tracking meaningful leading indicators. Most are still caught up in command-and-control management styles, with decision making far-removed from the real-time action on the ground.

Meanwhile, those seeking to change how they work encounter an array of misinformation, misguided “transformation” procedures, and ineffective models.

I went through my fifth McKinsey deck last week, which was literally the same business strategy for a different company in a different market.

I was speaking to the leadership team of a major bank, and they were showing me a table for their digital strategy. Then I happened to be talking with an insurance company halfway around the world, who’d also just had their strategy worked up. Both were created by McKinsey.

They were the same boxes with the same words! It was all literally copied and pasted, and it was boilerplate—grow customers, increase revenue, greater engagement, better workplace…unbelievable!

You could just lift this stuff out of any business journal, copy and paste—tick box innovation. There was nothing unique or customize performance metrics to the problems that these businesses are actually facing in their context.

When I asked the bank team what their biggest challenge was, they said, “We’re not seeing younger people sign up for new accounts with us.”

“And how are you measuring that?” I asked.

Their answer: total market share and revenue.

How is that going to tell you if you’re solving that problem? It might someday, but it will take months to find out. You need to find out in minutes.

Don’t Fall For Misguided Innovation Metrics

The internet is full of misinformation about innovation metrics —for example, the idea that your number of marketing campaigns is a leading indicator for an increase in sales is rubbish.

Counting the number of times you perform an activity is NOT the way to understand if customers find it valuable—and are willing to part with their money. Leading indicators are nothing your company does. They’re changes in customer behavior.

Your activity is the cause. What you need to track are the effects your efforts are having in the market.

“Your activity is the cause. What you need to track are the effects your efforts are having in the market.” Barry O’Reilly

Click To Tweet

You Must Think Beyond Standard KPIs

Another thing you can easily find on the internet is an abundance of standardized metric sets supposedly proven as indicators for business success. The idea is that these are the metrics smart companies use.

For example, one of the most widely circulated blog posts put out by Andreesen Horowitz offers a set of standardized metrics for SaaS startups.

These sets are very popular because people know so little about metrics. Everybody wants a template: “the seven metrics you need to track to be an innovative company.” There’s nothing innovative about that!

The point of tracking metrics is to guide your decisions in as close to real time as possible, based on your unique set of most pressing problems.

Common metrics provide a base set of KPIs that every business should monitor, but you need metrics that speak specifically to your situation:

• your most pressing problems
• the challenges you wish to solve
• the diagnosis you’ve made about gaps in a market
• and how you can know your proposed innovation is capturing it

The Most Common Mistakes in Metrics

Because of all the misunderstanding and misinformation, most companies find themselves hamstrung right from the start. Let’s look at where and how things are going wrong.

Companies Lack a Cohesive Success Tracking System

First, most companies simply don’t measure their innovation performance at all—what they measure, how they measure, and when they measure is all out of sync and not systematized.

What companies measure

Most are looking exclusively at solution-focused, output-based metrics—work completed on time, on budget, in scope. When someone has an idea, the first questions asked are, “How long is it going to take? And how much is it going to cost?” These tell you nothing about business impact.

How companies measure

It’s common practice to use unwieldy spreadsheets rather than true analytics tools. Companies struggle to collect, organize, and understand their data. So they can’t use it effectively to make good investment decisions, look for trends or direction of travel over time.

When companies measure

Most don’t have analytics in place from the beginning. They measure their results only after a product is shipped and live to customers. With this approach, you have to invest a lot of money before you actually get a measure. That’s not very helpful. And certainly doesn’t allow you to learn fast what’s working or not.

Companies That Use Metrics Focus on the Wrong Factors

A minority of businesses actually measure business outcomes, which are changes in real human behavior that lead to a business impact. This is a step in the right direction, but most are still not optimized.

• They’re tracking lagging indicators (e.g. revenue)—the last potential moment you could find out if what you’re doing is successful.
• They’re using absolute values (e.g. launch 7 marketing campaigns), rather than relative rates and ratios for improvements in performance (e.g. reduce wait time by 10%).
• They’re seeking perfect data, whereas they should be using best thinking and improving accuracy as they learn. They waste time debating statistical significance and fighting over the perfect number to represent success.

When I push companies on actual, quantifiable metrics, they have no answer. They say, “We need to be faster.” “We need to make more money.” “We need to grow our customer base.”

Everyone’s scorecards contain the same things: increased financial rewards, greater customer engagement, happier workers. Of course you want those things—who wouldn’t?

But none of those universal measures tell you anything about how you’re doing in your market with your customers.

The handful of companies who truly understand how to identify and track leading indicators of desirable (and undesirable) business outcomes wield a profound advantage in the market.

The rest are falling behind, and they’re left wondering why.

“The handful of companies who truly understand how to identify and track leading indicators of desirable (and undesirable) business outcomes wield a profound advantage in the market.” Barry O’Reilly

Click To Tweet

The Right Way to Approach Innovation Metrics

Where all the magic happens is leading indicators—signals that help you learn in the earliest possible moment if the changes in customer behavior you’re hoping for are happening. These are the signposts that tell you if you’re on the path to your desired outcome.

One way of thinking about leading indicators for innovation is this: it’s the “A-ha” moment customers really see the value of your product, and the speed, frequency, rate or ratio of people getting there. Let’s look at a few examples.

Robinhood incentivized people to get their friends onboard by giving them a new share. It tracked what percentage of people who were signing up for new accounts and trading were then encouraging other people to join. Then it determined a viral coefficient of how many users were bringing new people into the platform, and it optimized for it.

Facebook discovered that when new members connected with 7 friends within their first 10 days, they would stick on the platform. So it encouraged new members to connect with friends right away and constantly improve the percentage and rate of new members that did it. Slack found a similar effect for teams that sent at least 2000 messages.

Discovering these leading indicators enabled these companies to grow rapidly and become dominant players in their markets.

How to Find Leading Indicators

The sad reality for all the template lovers out there is that if you want to innovate, you have to get very good at diagnosing a specific problem either in the market or for the customer.

You have to understand that solving the problem well would result in some unique change in behavior that’s unique to that problem.

Therefore, if you’re trying to innovate, you have to define outcome-based measures for the problems that you’re solving. And you need to discover the leading indicators that can tell you if you’re achieving the outcome you’re aiming for, getting warmer, or totally off the mark.

Follow these basic steps to create your leading indicators:

1. Look at the problem you’re trying to solve and make sure you understand it clearly. Whose problem is it? What is their reality?
2. Figure out what would be the first signs of success in solving the problem (Hint: not revenue). The way to create leading indicators is to create stories—what could people be doing that would tell you early on if your idea is working or not.
3. Set up experiments to try to generate the desired effects and track your results.

More Success Tips for Innovation Metrics

Before I wrap up, I should mention a couple of other factors that will help ensure you track the right metrics and get the most benefit from your data.

Define Metrics for Success AND Failure

Many companies only talk about success. They implicitly assume you either hit the mark or you don’t—pass or fail in one go.

The reality is, your initial efforts might create a result that falls short of desires and expectations, but could still indicate your path has potential. Or, you could get a result indicating disaster ahead.

If you don’t define a metric for failure and measure against it, you won’t be able to distinguish it from partial success.

You need leading indicators for both success and failure to find out quickly if you’re moving in the desired direction OR the totally wrong direction. These are your guardrails for effective decision-making.

“If you don’t define a metric for failure and measure against it, you won’t be able to distinguish it from partial success.” Barry O’Reilly

Click To Tweet

Your Metrics Need to Work Together

Another mistake is defining metrics in isolation, rather than in relation to each other. You need tension in metrics so you don’t over-optimize.

Customer conversion is a classic example. You can increase conversion by dropping the price of your product, but profit will suffer over time. The trick is to use counter-metrics that help you increase conversion AND maintain or increase profitability. Having two counter-metrics is great, and three is amazing, but no more!

Mastering metrics and using them to guide investments is what true innovation is about. You must find the changes in behavior that leads to a business impact as a result of your work.

Here’s a challenge: go through the steps above and come up with three potential leading indicators, then start to track them. If you’d like, send me what you come up with and I’ll give them a look.

And if you’d like in-depth support for your innovation efforts, I’m here to help. Let's talk.

__________________________________________________________________________

Something Else To Read...

The growth of investment in digital transformation is compounding at 18% year-on-year. By 2023, an estimated $7 trillion will be spent on these initiatives annually. To put that in perspective, total US expenditure on healthcare for 2020 was about $4 trillion—during a pandemic.

Companies are spending a fortune on these efforts, but are they good investments? Often, the sad answer is no.

80% fail, most often due to lack of alignment and understanding among leaders on how to execute against a digital transformation strategy. Only 16% of employees said their company’s transformations have improved performance and are sustainable.

Read more

__________________________________________________________________________

?People AND Automation with Nancy Householder Hauge, Chief HR Officer at Automation Anywhere

Nancy has won the Stevie Award for women in hi-tech and is considered one of the most influential women in Silicon Valley. On this podcast, you will discover how she spots future disruptors, the importance of thinking time, and why automating human jobs should be celebrated rather than feared.

Decentralizing Data: From Data Monolith to Data Mesh with Zhamak Dehghani, Creator of Data Mesh

Zhamak is currently the Director of Emerging Technologies at ThoughtWorks and the creator of Data Mesh, which I describe as “one of the most exciting paradigm shifts in how we manage data at scale.” This podcast is going to teach you why traditional data architecture models are failing and how applying product thinking principles to data management is a way to harvest the data’s full potential.

Want to absorb these kinds of ideas on the go and hear from industry leaders?

Listen and Subscribe to the Unlearn Podcast at

Apple Podcasts | Stitcher | Google Play | Google Podcasts | Spotify | iHeartRadio

__________________________________________________________________________

Upcoming Events

How American Airlines is Transforming Its Culture by Transforming Delivery with Joanne Montesano and Jason Hobbs

Thurs. April 8th - 12:30pm PT | 2:30pm CT | 3:30pm ET | 8:30pm GMT

Join this livestream for a deep dive on case studies with two front line leaders from American Airlines, Joanne Montesano and Jason Hobbs on what challenges they had to tackle, the hard lessons learned and extraordinary results realized as AA transformed it's culture by transforming delivery.

Watch Here