Are you measuring the right Product Management metrics??
Thomas (Tom C) Chmielewski
Vice President of Product Management - Improving Existing Portfolios, and Designing & Launching New Products & Services
Measuring your product performance is vital for determining the health and success (or failure) of a product and is critical in determining continuing investment(s) decisions.
The measurement of the product performance should be a blend of metrics in order to appreciate the overall product health. For example, sales may be growing but support costs may be growing faster. And the weightiness of the metrics will change over the course of the product life cycle.
For example, gaining market share may be more important at the start of the product life and customer retention may be more important towards the late stages of the product life.
As a product manager, you are responsible for keeping the current, meaningful, report card on your product.
The metrics can be broken down into four categories: Financial, Approval, Churn/Sales, and, Execution (F.A.C.E.).
Financial:
No matter where your product is in the lifecycle, revenue is always key. Even for non-profits revenue is key to ensure they can cover expenses and remain operating as a non-profit. Revenue is king.
· Revenue – the amount of revenue generated by the product but also including additional service revenues such as consulting, training, and support (sometimes the revenues are separated out)
· Cost – the amount spent on building the product (and delivering services)
· Profit – the gross profit P is the difference between the cost to make a product C and the selling price or revenue R. P = R - C.
· A single number is just a point, so compare the numbers to where the product was last month/quarter/year, and where you expect to be next quarter/year to trend your progression.
· Average Deal Size – Is your revenue made up of a few very large deals or hundreds of smaller deals? Is the deal size trend growing or shrinking?
· MRR Monthly Recurring Revenue – some products are one time sales, some products are monthly subscriptions, and some are a blend of the two. Knowing the-one time revenue versus the recurring revenue is important because it helps forecast future financials.
· ARPA, or Average Revenue per Account – ARPA is the worth of monthly “contract” with the customer or user. To put it simply, this is how much money an average paying customer gives you every month. You can use this further to determine which customers are profitable and which ones you are losing money on.
· For further refinement, break down revenue by geography and/or by vertical market.
Approval:
The market is littered with failed products. Even if you are currently selling a lot today, your product will not live long if your customers are dissatisfied. Negative reviews on social media can kill even a half-way decent product. Use several tools to measure customer approval – satisfaction or dissatisfaction.
· Net Promotor Score (NPS) –The Net Promoter Score (NPS) tells you the percentage of your customers who would recommend your product. “On a scale of zero to 10, with 10 being the highest, what’s the likelihood that you would recommend our company to a friend or colleague?” Customers who respond with a score of 9 or 10 are called promoters, while those who provide a score of 6 or lower are detractors. The formula is NPS = % Promoters - % Detractors. If 80% of your responders are promoters and 10% are detractors, your NPS is 70%.
· CSAT The Customer Satisfaction Score – CSAT is the average satisfaction score of customers influenced by a particular experience. The question is typically worded like this: “How satisfied are you with X?” Users can then select from a range of options from “Extremely Satisfied” to “Extremely Dissatisfied”
· Number of customer escalations – how many times do customers escalate to the sales or support, or executive teams? This is a key dissatisfaction indicator.
· Number and types of calls to the support group (defects, bugs, complaints, How do I?) including trends, duration, severity, repetitiveness
· Number and amount of required discounts or refunds to customers. If you are refunding on a regular basis you need to investigate where and why there is a problem and correct it.
Churn; Customer, Sales:
The revenue follows the selling so when sales dry up revenue dries up. Having an appreciation of the sales funnel and activities related to Sales allows the product manager to respond with product/price/promotion adjustments to ensure selling remains strong. How many sales leads does your product take to generate X qualified prospects to deliver Y demos to get Z contracts signed? Knowing and measuring these metrics enables you to help keep the selling (and revenue) strong. All of these are tracked by time-frame and trended.
· Number of generic customer leads
· Number of qualified leads / live customer prospects
· Number of demos requested/delivered. If the number of demos drops and typically your customers sign a contract three months after demos, you know you will have a revenue issue in the upcoming three months.
· Number of proposals issued, the value of proposals issued
· Number of closed deals, the value of closed deals
· Length of time to close a deal from start to finish
· Time from contract signing to go-live
· Win / Loss analysis - why did you win and why did you lose (DO NOT rely on Sales, perform the analysis yourself or assign to an independent agent)
· Measure these in total and by geography or vertical
· CAC, or Customer Acquisition Cost - acquisition marketing costs divided by the number of paying customers acquired over a period of time. I.E. How much are you spending to acquire customers?
· Churn Rate - Percentage of customers who stop using your product every month. Divide the number of customers lost in a month by the previous month’s total customers. It is a good indicator of product health and is absolutely critical for monthly subscription products.
· Contract renewal rate – how many customers renew contracts versus abandon your product
· Market penetration rate – what percentage of the market does your product have versus competitors and is your share growing or shrinking?
Execution:
Who is using your product? (Are they using your product?) How are they using your product? What features are they using or not using? Because if no one is using your product you won’t get renewals, you won’t have referrals, and eventually, you won’t have revenue.
· MAU Monthly Active Users
· Number of active users per day
· Number of models/jobs/actions run per day
· Number of files ingested/used/processed
· Number of records processed
· Number of sessions per user
· Session duration
· Number of key user actions per session
· Feature usage (usage vs other features)
· Dwell time for web apps
· Abandon rates for web apps
There is also the opportunity to measure the Marketing/visibility for the product and how fast did you reach the market with your new product release(s) compared to last year?
· Number of times mentioned in the press/press coverage/press releases
· Mentions or coverage or presentations at conferences/tradeshows
· Internal visibility
· Number of courses taken, number of training sessions given
· Creation of a User Group and associated participation
· Number of times materials were requested/used by Sale
In the absence of a current scorecard pick just five to seven key metrics and start measuring and trending those. That will give you a good start into the insights of your product.
You should manage what you measure so be very careful in the metrics you select as prime metrics. For example, making customer satisfaction a prime metric may lead to impacts in other metrics such as an increase in contract discounting and an increase in adding on free services.