How do you ensure your product's metrics reflect its true value?

As a product leader, I always ensure that our metrics truly reflect the value of our product. The first step is understanding the product's value proposition - what it offers to customers that others do not. We then define 'value' in terms of specific outcomes and behaviors we expect from users. When choosing metrics, we focus on those that align with our product's goals and vision. For instance, for a SaaS product, we might look at Monthly Recurring Revenue (MRR), Churn Rate, or Average Revenue Per User (ARPU). We avoid common pitfalls like vanity metrics that look good on paper but don't indicate real growth or success. Instead, we opt for actionable metrics that can guide our decisions and strategies. Data-driven insights are crucial.

There is always nuance with metrics, but extrapolating through these 4 lenses is helpful: (1) input metrics, (2) output metrics, (3) customer value metrics, (4) business value metrics. Differentiating input vs. output metrics allow you to understand correlation, or ideally, causation. An example could be feature-level engagement (input), user retention (output). Differentiating customer value vs. business value allows you to have more nuanced conversations about the performance of your product/business. Ideally these trend in a linear fashion, but there may be times where you're focusing more on one vs. the other and having separate metrics to facilitate those conversations is helpful.

I would also add that while metrics like DAU, or conversions are super popular to track and often work at the macro ultimately good PMs tailor their metrics to their context, and in turn their narrative. “What truly helps me understand if we are creating and sustaining value with X,Y is Z - etc.” Lastly I think business are super shy on key metrics - these should be super visible and super open for debate. The only way you truly avoid vanity metrics is by validating them against as many minds as possible. It’s easy to lose sight of the difference in true value between count of sessions and count of users if only one voice is inputting on what metrics should be used.

Begin by understanding your product's stage and goals to categorize and prioritize metrics. For instance, in the acquisition phase, focus on metrics like visit-to-signup ratio. Differentiate between vanity metrics, which may look good but lack depth, and valuable metrics that directly impact business objectives. Causative metrics drive an outcome, correlative metrics relate to an outcome, while contributing metrics support an outcome indirectly. Develop a robust tracking system, ensuring it's timely and consistent. This enables accurate analysis and trend identification. Use these insights to guide product development, decision making. For example, if the churn rate is high, it might be time to improve user experience or customer service.

Determine Needs: Identify what you need to measure, aligning it with your organization's goals. For instance, if user growth is the goal, track new sign-ups. Select Metrics: Choose specific, measurable, achievable, relevant, and time-bound (SMART) metrics. Establish Calculation Methodology: Define how you'll calculate metrics. For example, churn rate might be calculated as: (Number of customers at the start of the period - # of customers at the end of the period) / customers at the start of the period. Define Terminology and Formulas: Ensure everyone understands the metrics used. Document and Communicate: Share metrics definitions with stakeholders. Consistency: Maintain uniformity in measuring and reporting to ensure data integrity.

Let's say we're measuring Active Users, Churn Rate, and MRR for our SaaS product. Firstly, define clear standards for each metric. Active Users might be those who log in at least once a week; Churn Rate could be customers who cancel their subscriptions within a given period; MRR would be the predictable revenue from subscriptions. Next, develop a testing framework using tools like Google Analytics or Mixpanel. Set up A/B tests to measure different user behaviors and responses to changes. Simulate changes like new feature releases or pricing adjustments. Monitor if your metrics respond as expected. Refine your metrics based on these tests. If a metric isn't reflecting desired outcomes, it may need redefinition.

Identify Current Metrics: Start by reviewing the metrics you're currently using, such as active users, churn rate, or MRR. Evaluate Metrics: Assess if these metrics align with your product goals. If your goal is user growth, but you're not tracking new sign-ups, there's a disconnect that needs addressing. Monitor and Analyze: Keep an eye on performance trends. If churn rate is increasing, it's time to investigate and address potential issues. Share Findings: Communicate metrics and their implications with your team. Their insights can help optimize your metrics and improve your product. Remember, metrics are not just numbers; they're tools for understanding your product's performance and guiding its growth.

As a product manager, it's essential to remember that while metrics provide valuable insights, they're not the whole story. For example, your metrics might show a surge in new users, which seems positive. But deeper analysis might reveal these are low-quality sign-ups with high churn rates. In this case, focusing solely on increasing sign-ups could harm long-term growth. Metrics should always be interpreted in context and used as a tool, not an end goal. Regularly revisiting and refining your metrics ensures they continue to provide meaningful insights as your product and market evolve.