The Paradox of SaaS Metrics and Benchmarks

Please take a moment to become part of the industry’s largest SaaS metrics benchmarking research by participating in the 2024 SaaS Performance Benchmarking Survey by clicking here

I read a LinkedIn post on Friday that basically said being beholden to SaaS Metrics, in this case “Net Revenue Retention” is a mistake. The post provided the context of closing a new deal that loads all expected expansion over the next five years into the first year ARR and increasing the cash received upfront, but at the expense of having no expansion ARR for the next 4 years thus making NRR at 100%.

The post and the related comments made me think long and hard about how SaaS Metrics and Benchmarks used incorrectly, such as to promote an individual point of view, make another department look bad or purely to satisfy investors can be misleading and dangerous to a company’s financial health.?

As I considered how to write this week’s newsletter on the above topic, I decided to use a Top Five structure to frame the newsletter content and provide some best practices when using SaaS Metrics and Benchmarks to inform decision-making.

Top Five Practices for Using SaaS Metrics and Benchmarks

1?? Start with a Holistic SaaS Performance Metrics Framework

I introduced Benchmarkit’s SaaS Performance Metrics Framework last week, which I will not repeat in detail, but below are a few components of a SaaS Metrics Framework:

• Start with the top 5 company-level metrics (outcomes) for the fiscal year

• Break down each of the company-level metrics into their component parts
• Identify those input variables (metrics) that have a causal relationship to the company-level outcome metrics

• Assign the company-level outcome metrics as SHARED goals to each department head that can impact the metric creating a culture of shared responsibility to the outcome metrics that are most important to the company

• Assign the input variables (metrics) as a goal(s) to the department head responsible for that metric and the associated processes and resources

• Calculate and communicate the metrics trends via company wide dashboards that highlight how the input metrics and outcome metrics are trending?

Below is an example of a company-level outcome metric (Net Revenue Retention) and its component parts.

NRR Goal = 110%

Net Revenue Retention (NRR), also known as Net Dollar Retention (NDR), measures the percentage of recurring revenue retained over a specific period. NRR captures lost ARR due to customer attrition, reduced usage, or decreasing subscription level, offset by increased revenue from existing customers through up-sells, cross-sells, price increases, or increased usage. The term “net” is used because lost revenue is “netted” against expansion revenue.

NRR is directly impacted by two primary input variables:

• Gross Revenue Retention (GRR)
• Expansion ARR

Gross Revenue Retention (GRR), also known as Gross Dollar Retention (GDR) measures the percentage of recurring revenue that is retained over a specific period of time. GRR captures lost recurring revenue due to customers churning or reducing their subscription commitment.

Expansion ARR consists of organic expansion, up-sells of the current product or cross-sells to additional products in current customers resulting in increased Annual Recurring Revenue (ARR).

Gross Revenue Retention and Expansion ARR should be assigned as goals to the function responsible for each.? Often, GRR will be the responsibility of Customer Success, and Expansion ARR will be the responsibility of a combination of Customer Success to identify an expansion opportunity and an Account Manager or Account Executive in Sales responsible to qualify and close expansion opportunities.

In larger organizations, there is an opportunity to align a third level of input metrics that have a causal relationship to GRR, the second level input variable (metric) from the above example.??

Examples of third level input variables (metrics) that could be assigned as goals (metrics) that directly impact GRR include:

• Product Utilization
• Customer Satisfaction Score
• Customer Health Score
• Net Promoter Score
• Customer Verified Outcomes

2?? Develop Standard Definitions and Calculation Formula for Every Metric Used

A common challenge for many companies in the use of metrics is not having a common, standard definition of the metric, including the calculation formula. This lack of standardization and usage becomes exacerbated when multiple functions are using the same metric, but for different purposes.

One approach to ensure alignment across the company is to create a “Metrics Council” composed of representatives from Finance, Marketing, Sales, Customer Success, Revenue Operations and Product.? The council has the responsibility to define, document, publish, communicate and maintain the standards for how metrics are calculated and used in the company.

If you create a “Metrics Council”, it will be valuable for them to conduct educational sessions for all employees to share the definition of the metrics being used across the company, how they are being used and why they are valuable

3?? Instrument, Calculate, Publish and Communicate Metrics on a Regular Cadence

If metrics are a foundation for measuring company performance, to capture signals that are predictive to future outcomes, and are used as key inputs to decision-making, it is important to automate the capture, calculation and communication of the metrics

The latest Benchmarkit research shows that more than 65% of companies use manual processes and spreadsheets as their primary SaaS metrics calculation tool.? This approach will become resource intensive as the company scales, the number of source data systems increases, and is extremely error prone, especially when multiple people are responsible for the calculations. As such, investing in the automation of SaaS Metrics source data capture and calculation is a great investment that will yield high returns in a metrics-driven decision making culture.??

A key point is to invest in the infrastructure to automate the process sooner than later.? Beginning the metrics automation process as a company progresses to $5M ARR and above is a good time to start.

As with any strategic initiative inside a company, it is critical to consistently communicate how the company level and second level metrics are performing.? Having a regular monthly or quarterly communication cadence to share the metrics trends, re-inforce how they are being used in the decision-making process and highlight the positive and potential negative impact of those trends.

4?? Compare Internal Performance Metrics Against Similar Like Company Benchmarks

I compare benchmarks to weapons of mass destruction.? Placed in the wrong hands, used haphazardly and used to promote discord with other departments -? benchmarks can be misleading and dangerous.

The first rule is to use benchmarks that are based upon like companies that share in your company profile attributes including:

• Company Size (ARR or Revenue)
• Average Annual Contract Value (ACV)
• Distribution Model (Product-Led Growth / Sales-Led Growth)
• Pricing Model (Subscription, Usage-Based, Hybrid)
• Financing Source (VC, PE, Bootstrapped, etc.)
• Product Category (Horizontal App, Vertical Industry, Infrastructure, Cyber, etc.)

Not all benchmarks reports and datasets consist of a large enough population to be segmented by each of the above company profile attributes.? Whenever possible use benchmarks that are at least segmented by company size and ACV, as they are often highly correlated to the relevance of the benchmarks.

Benchmarks are best used as “directional guideposts” versus “absolute goals”. Understanding the trade-offs between different metrics and the associated benchmarks is an important step in using benchmarks.?

Another risk in setting an absolute goal, such as using a NRR benchmark for companies in your ARR range, is that the benchmark may not factor in the impact of the pricing model (Usage-Based vs Subscription), the number of products available to cross-sell, or when the categorization of New ARR ends and Expansion ARR begins.

5?? BEWARE Over Indexing on a SINGLE METRIC and/or BENCHMARK

This recommendation? was a primary reason we created the SaaS Performance Metrics Framework.? Increasing enterprise value is not dependent upon a single metric.? Though growth rate is still the number one factor impacting enterprise value to revenue multiples, other metrics including the Rule of 40, Net Revenue Retention, and CAC Payback Period have high enough correlation (as measured by R-Squared) to enterprise value that they cannot be ignored.

One practical example is that the composition of the two different variables used in calculating the Rule of 40 is material to the resultant enterprise value to revenue multiples. Using a regression analysis on public Cloud companies Rule of 40 score, those with a 30%+ growth rate and at least a 10% Free Cash Flow Margin exhibit much higher EV:NTM Revenue multiples than companies with a 50% growth rate and a negative Free Cash Flow Margin.

Another example might be the acceptance of a CAC Ratio in the lower quartile as a trade-off to achieving growth rates in the top decile.

A more “nuanced” example might be over indexing on new customer logo acquisition velocity by expanding into new markets, even if the customers in those markets have a much higher churn rate which reduces both GRR and NRR.? This is an example of optimizing for a single metric (logo acquisition velocity) at the expense of other important, downstream performance metrics…especially when the negative impact is a delayed resultant outcome.

A final note of caution on using metrics and benchmarks is highlighted by “Goodhart’s Law”.? What is Goodhart’s law?:

“When a measure becomes a target, it ceases to be a good measure”

Goodhart’s law was first used to critique monetary policy. The concept effectively says that when we set one specific measurement as the goal, people will tend to optimize for that goal regardless of the consequences.?

Goodhart’s law is a good reason to understand that SaaS Performance Metrics are highly instructive when used properly, but can be misleading and even dangerous when not used in context of a holistic understanding of the multiple input variables that directly impact any single metric.? No single metric can stand alone in an organization which is made up of multiple interconnected, interdisciplinary functions.???

Most importantly, SaaS company value is not determined by any single metric. As such it is critical to take a more comprehensive approach using a holistic, integrated metrics framework and approach to using metrics that reflects the reality of enterprise value being created and calculated based upon multiple variables.

For the love of SaaS Metrics!!!