Hierarchy of Product Metrics

What metrics should you care about for your product?

The “Hierarchy of Product Metrics” is my attempt to synthesize insights from two product thinkers:

• Shreyas Doshi (who I am proud to call an early mentor) has an extremely thoughtful guide to product metrics that argues for tracking across a host of specific categories.
• Moe Ali taught me in his excellent Product Faculty course that retention is the “keystone” metric, without which everything else can fall apart.

This hierarchy is inspired by Abraham Maslow’s famous “hierarchy of needs” , which posits that people require “lower-level” needs such as food and water before they can meet “higher-level” needs like friendship and intimacy. In the same way, I believe that products must meet a minimum bar of the lower levels of the Hierarchy of Product Metrics pyramid before they can succeed at the upper levels. I’ll explain more as I explain how I define each level:

?? Health & Reliability Metrics

As Shreyas puts it , “Is the product available and performing in the manner that users would reasonably expect?”

• Sample metrics: uptime, latency times, load times, error/crash/failure rates, bug report volumes/severity, security/privacy incidents, spam/abuse rates
• Without a certain level of technical health and reliability, it’s impossible for your product to be successful. Note that you don’t need perfection here (e.g. “six nines” availability) before you can worry about “higher” levels—it’s just that without the basics, you’ll likely struggle to deliver on any of the other metrics.

?? Satisfaction & Happiness Metrics

How well are you solving users’/customers’ problems? How happy are they with your product?

• Sample metrics: NPS (Net Promoter Score), CSAT (Customer Satisfaction), CES (Customer Effort Score), other surveys/interviews, reviews (e.g. in app or product marketplaces), customer service channel metrics (e.g. support ticket volume/severity), social media/listening metrics (including sentiment analysis)
• There might also be some overlap with usage/engagement metrics such as task completion rate (see below)
• There’s a basic level of satisfaction that you need before you should start driving a user adoption funnel, particularly because satisfaction and happiness are the main drivers of retention.
• Note that some of your best data here might be (or at least start out) qualitative…I’m especially fond of ways to shorten the feedback loop like user testing with mock-ups or “wizard of oz” prototyping.

User & Growth Metrics

This is a broad bucket of metrics that track how many users you have, how that number is growing (or not!), and how they’re using your product. Most of these are what Shreyas calls Adoption metrics . I've divided them into Acquisition, Retention, and Engagement:

? Acquisition Metrics

How many new users/customers are you getting? How many “convert” or “adopt” to become “activated” (or paying) users?

• Sample metrics: new user acquisitions (by source, over time, etc.), sign-ups/downloads/installs, conversions from free to paid/upgrades, CAC (Customer Acquisition Cost), adoption/activation/acquisition rates and funnel metrics (and feature adoption rates), traffic sources, lead generation rate, TTV (Time to Value)
• Note that even though I’ve listed Retention as the “keystone” metric, you need at least some new user acquisition to stay afloat since it’s impossible to retain 100% of your users. Your product’s growth is driven by both acquisition and retention.
• Note that while retention is often “product led”, it’s harder to have acquisition be purely from the product itself and organic “word of mouth”.

?? Retention Metrics

How many users/customers stay over time? How many are lost or re-acquired?

• This is the keystone around which all the other user metrics flow (hence the ?? on my diagram). It doesn’t matter how many users you acquire if you are unable to retain them (...at least for most businesses; only a handful have customer acquisition costs lower than retention costs). Similarly, even if your users are very engaged when using the product, if you aren’t able to retain them, eventually you’ll have no users. This isn’t to say that acquisition and engagement don’t matter—it’s just that you need a reasonable retention curve to make investments in those areas stick.
• Sample metrics: retention curves (consider both unbounded/rolling & N-day retention), cohort curves/retention rates, churn rate (and cancellations/downgrades), renewal/reactivation rates
• Note that retention is often driven by satisfaction & happiness, which can help answer “why” questions like “Why did retention drop in the past month?”. Also consider segmentation—for example, if you’re struggling to retain your female or Black or Spanish-speaking or visually disabled users, you have a problem.

?? Engagement Metrics

How much (and in what ways) are users/customers using your product?

• Sample metrics: DAU/WAU/MAU, MAU per install, DAU per MAU (“stickiness”), CTR/bounce rates, changes over time (e.g. M/M or Y/Y), # sessions per day/week/month, and various things per day/week/month/session: time spent, actions taken (likes, shares, comments, etc especially if they’re part of the “engagement flywheel”), # pieces of content consumed, task completion rate / time to complete tasks
• Often so-called “north star metrics” are in this category, like Watch Time for YouTube, Bookings for Airbnb, or Rides for Uber. While such “north star” metrics are highly correlated with user value and thus growth, myopic fixation on any single metric is usually sub-optimal.
• Also consider more fine-grained metrics on usage patterns, what Shreyas mostly calls “Usage metrics” , things like what times of the day or week usage spikes, which features are used in what ways, whether/how people engage with help documentation, etc. Only focusing on the top-level numbers can obscure useful/important data.
• Note that you don’t have to optimize (just) for raw engagement—my former colleague Gene Yoon has some interesting ideas about tuning social media content recommendation ML to help users gain empathy, for example. (It might be interesting to optimize systems directly for user happiness…this might achieve product and business objectives better than engagement in the long run?)

?? Business Metrics

Is the product meeting business objectives? Shreyas calls these “Outcome metrics” (In many cases these are about revenue and profit, but there could be other objectives—for example “number of lives saved” by a medical product.)

• Sample metrics: gross/profit margin, revenue, CLV/LTV (Customer/User Lifetime Value), ARPU/ARPA (Average Revenue Per User/Account), ACV/TCV (Average/Total Contract Value), AOV/RPV (Average Order Value/Revenue Per Visitor), NRR/NDR (Net Revenue Retention/Net Dollar Retention…which are arguably both Retention & Business metrics), MRR (Monthly Recurring Revenue), ARR (Annual Recurring Revenue or Annual Run Rate), cost reduction and efficiency metrics
• Also consider the Ecosystem (Shreyas separates this into a separate category ), which can be a leading or lagging indicator for your main business metrics, things like market share, share of TAM and/or wallet, brand awareness, integrations

Closing Thoughts

To give one example of how these metrics can flow into each other, when I was at Google, experiments showed that improving Google Search latency (Health & Reliability) led to more searches per day/session (Retention & Engagement), which led to higher revenue (Business).

Metrics depend on the context of your product, its market, whether you’ve found product-market fit, etc…but I’m convinced that these broad categories and their relationships are true for the vast majority of products.

What do you think? Please leave a comment if you have something to add, especially for how this framework might be improved.

#productmetrics