NPS Weighting—Should You Do It?
Ryan Stuart
CEO, Founder at Kapiche | Understanding human feedback at scale | We're hiring! ??
This is a summary version of a blog article that appeared on the Insights Hub - a space for customer insights professionals to grow and develop their career in Insights. You can read the full article here.?
I’ve lost count of the number of times I’ve been asked to weigh into an internal debate a Kapiche customer is having about whether they should be weighting their NPS score (similar questions apply to other CX metrics as well). Weighting your NPS (or other CX metric) means applying weights to the score such that the score is representative of your customer demographics.?
Reflecting on those discussions, I’ve come to realise that there are good arguments on both sides of the fence. I’d argue, if you’re trying to make a choice between weighted or unweighted NPS numbers, then the answer is always to track both.
So, Weighted or Unweighted NPS?
Both! There are a bunch of reasons why, but one of the most important is so you can see when there’s a big difference between unweighted/weighted NPS (or any other CX metrics). If there is a large discrepancy, this could indicate one (or a few) different issues are at play. Here are a few possible reasons why that might be the case:
Insights teams are judged by their ability to provide high impact insights to decision makers. These insights are used to make decisions regarding what to do next. Relying solely on unweighted data could jeopardize your ability to provide quality insights.
领英推荐
For example, let’s say you have a customer base that’s 90% women and 10% men, but your sample rate somehow includes 50% women and 50% men. From looking at the unweighted NPS response data, a significant insight might be: ‘Something’s driving the NPS down by 5 points, and it seems to be localised to men. Let’s do something about that to improve our score by 5 points.’ And so, your company could put more effort towards addressing its male customers.
Conversely, there could be something in women’s eyes that’s bringing your score down 3 points. For example, you could discover a valuable insight that women really dislike poor customer service from frontline staff, and it is? causing a 3 point decline in unweighted NPS.
On the surface of it, if you were to rely solely on unweight NPS, you would prioritise actioning the insight from the male responses, because they are having an impact of 5 on the NPS score. However, if we were to weight that impact, it would reduce to 1 point, because only 10% of our customers are male. The weighted impact of the insight we identified about our women respondents is 5.4 compared to the unweighted impact of 3. Armed with this information, we should definitely prioritise addressing the insight we found about women!
Simultaneously to this, we should also be asking ourselves another question: Why is our survey sample so drastically skewed towards responses from men? They make up 10% of our customers but 50% of responses to this survey. It could indicate that women are far less engaged with our product or service, that there is an issue with the survey distribution, or that there is something specific to this touchpoint in the user journey that is causing this phenomenon that we should be aware of. Either way, it warrants more investigation.
When to use Weighted NPS
Personally, I always start with analysing the raw unweighted NPS data. I’ll then use the weighted NPS score as a sanity check for the impact or veracity of any insights I’ve found. That way, I’m ensuring the insights I present to the business are the most impactful. In addition to this, I’m constantly on the lookout for large discrepancies between the weighted on unweighted NPS scores. Investigating these differences can themselves produce some valuable insights.
This is a condensed version of an article originally published on the Insights Hub. You can read the full article here.
Market Research | Customer Insights
1 年Hi Ryan, very interesting article, and interesting perspective. There are some cases however where the unweighted score would be very different from the weighted score, for example if you have NPS across a number of products or brands within your portfolio, and some of those products/brands have larger market share and hence larger share of the customer base. Example, if brand A has 80% of customers and Brand B 20% of customers, if your sample base reflects this with 80% brand A respondents then the unweighted NPS of both brands would give you a skewed perspective whereas a weighted score would be heavily skewed by the 80% customer base. So perhaps the question is whether taking a weighted average would be a the best benchmark as it would put more focus on the bigger part, or whether brands/products in the portfolio should be given equal weighting and importance when creating this benchmark?