Measurement mindset in product management - identifying the right KPIs

Most product teams understand the benefits KPIs produce, but identifying the right ones is a challenge that most struggle with. This article reflects upon key guidelines and tips helpful to create acceptable KPIs and a success metric dashboard.

A product manager’s role can essentially be summarised as following:

• Respecting customers and their voices - by knowing who the customers are and what problems do they have
• Helping customers being awesome - by giving them a product that meaningfully solves their problems
• Figuring out what game a company is playing and how to always be ahead - the Strategy, Competition & Commercial viability.

Managing a product is not an elementary task though. It spans over numerous sub-processes, challenges, and dynamic forces, that involve thorough customer research, iterative product development, implementing a scalable architecture, in a constantly changing market with shifting customer expectations, varied stakeholder opinions and interests all within a significant time horizon. 

Altogether these factors have tremendous potential to divert the product from its strategic long term goals or make it fairly difficult to stay on the right path.

In this context, KPIs or success metrics are handy tools for product teams. They act as a guiding light to help them stay on the right track in regards to their vision and strategy.

Key Considerations for identifying KPIs

As a rule of thumb, there are two considerations for identifying KPIs:

1. Using a holistic view
2. Keeping it lean

1. Having a holistic view

This fundamentally means measuring KPIs at all levels and in all areas.

Measure at all levels

As much as your product strategy directs the KPIs of your product, releases, and richness of features, your product will also be impacted by sprint goals, epic success metrics, and stories feedback. A constructive measurement framework is fundamental to provide direction at all levels. 

Strong customer feedback on the release of an epic may result in significant changes in your roadmap features which in turn might require tweaks in future releases and the overall strategy. Simultaneously, your strategy directs which features go on your roadmap and the richness of features on your roadmap guide your backlog items. Hence, why it is crucial to measure KPIs at all levels from strategy to epics

Measure all areas

Measuring every aspect of your product development process is essential. Measurement insufficiency in any area has a cascading impact on the success of the product. 

For example, your previous release might have increased the number of active users of your product and supported your revenue targets. But limitations on a healthy release pipeline, an ineffective regression suite, a lack of management sponsorship or weak team motivation will make it impossible to keep the momentum.

2. Keeping it lean by avoiding vanity metrics

A measurement framework can be costly. The more complex it is, the more time and energy your team will spend discussing and making decisions on it. Hence why it is absolutely mandatory to keep it lean, specific and avoiding vanity metrics.

I was once working on a retail product for a company that was originally selling its goods on major e-commerce platforms (e.g. Amazon), which meant that a substantial fee was paid to such platforms on every item sold. Although it was enticing to measure customer acquisition on the new retail product we were building, this KPI alone wasn’t a credible measure of how we were performing according to our strategy. Instead, measuring the conversion of customers from other e-commerce platforms to the new retail product was of higher value. It accurately translated the reduction in fees paid to other e-commerce platforms and reduced the customer acquisition costs burden.

Other guidelines for maintaining the right metrics 

In addition to the key considerations, some additional guidelines that are helpful for choosing and maintaining the right success metrics: 

Business goals direct success metrics

Use business goals to choose the most authentic KPIs, as those business goals provide the definition of success

Tie metrics to PLC (product life cycle)

The product lifecycle is an excellent tool for managing strategies of a product in each phase. It is staggeringly beneficial to use KPIs for managing decisions and objectives in various phases of the PLC 

Know your audience

It is extremely important to be aware of your audience and the appropriate manner of sourcing feedback. Especially, customer and stakeholder feedback

Right measurement frequency

It isn't fruitful to measure all KPIs at the same frequency. While commercial KPIs indicate trends over medium to long term horizon, user activity KPIs should have a much lower frequency. Choose the right frequency to capture and discuss your KPIs based on your product maturity and the metric itself  

Utilise all avenues

Especially for qualitative KPIs. Customer service, Marketing, and Sales teams are incomparable channels to hear the customer’s voice

Use qualitative and quantitative KPIs

Qualitative data through surveys or feedback captured via integrated tools like Optimizely speak volumes about user delights and frustrations. Quantitative data helps with efficient decision making. Balance them off appropriately at regular intervals

Benefits of a measurement mindset

The benefits of a measurement mindset are manifold. They not only guide you through the product path but also help you constantly adapt the path as conditions change so that you are always heading north.

?Avoid cognitive bias

It is easy to take pride in your work and let emotions steer your actions. Sometimes common trends in market which might not necessarily be a fit for your product forms a cognitive bias in your mind. KPIs helps get rid of cognitive bias

Ignore Hunches and Feelings

You will find ample people on your project who urge to make decisions based on "Feelings", "Gut", "Beliefs". Use data to lead them on right path

Suppress "Hippo's" 

Using data points is a guaranteed strategy to base your rationale and suppress Hippo's (highest paid person's opinion) 

Turn Failure into opportunity

Using data provides you an opportunity to fail early and learn early

Avoid getting lost into technicalities

Supporting your discussions with data points helps you bring back the focus on strategy instead of getting lost into minute details

Using data, information and empirical evidence is essential for making decisions and selecting the right approach. Hence, KPIs and a robust measurement framework that are unique to its product’s needs are core to the modern product management and development practices