Are Your Customers Truly Satisfied, or Is Your Perception Skewed?

I came across this interesting meme today. While I found it hilarious, It reminded me of an experience working with an organization a few years back.

I had just been hired to work with an organization struggling with poor CSAT and NPS ratings. When I join new organizations, I always review available data from their past surveys, complaints, and social media sentiments to gain context on how customers feel about the brands and their offerings.

When I reviewed this organization's data, I found out that the customers were dissatisfied with their experience with the business from their feedback in the previous year's annual survey, and the complaints data revealed many problems. Social media sentiments were not positive, and most had a theme around "unresponsive customer service."

Interestingly, when I looked at customer feedback from the customer service touchpoint, it had very positive ratings. I struggled to understand how this could be possible. The ratings for customer service touchpoints were disproportionately positive despite evidence of poor customer service performance evident in the annual survey and social media sentiment.

This contradiction prompted me to thoroughly investigate the survey methodology to understand which customers were sent surveys, when the surveys were sent, and how the feedback was analyzed.

When I asked the Customer Service Manager about this, a concerning revelation surfaced. They had intentionally excluded customers who had registered complaints from receiving post-interaction surveys. Their rationale?

"Why solicit feedback from dissatisfied customers?"

This deliberate omission highlighted a critical flaw in their approach to customer feedback analysis. The customer service team unwittingly skewed its CX metrics by selectively excluding unhappy customers, painting an inaccurate picture of customer satisfaction.

I educated the customer service manager on the importance of soliciting feedback from all customers, regardless of their interaction outcome, to ensure a comprehensive and unbiased understanding of the customer experience.

A sampling bias like this in VoC (Voice of the Customer) programs will lead to an inaccurate portrayal of customer satisfaction levels by skewing survey results toward positive feedback. This misrepresentation can mislead decision-makers into believing that the customer experience is better than it is, leading to complacency and neglect of genuine improvement opportunities.

The ripple effects of this misrepresentation can lead to erosion of trust and customer churn as the customers might seek out brands that demonstrate a genuine commitment to listening and responding to their needs.

That experience got me questioning how many organizations might be having such a problem.

If you have a high NPS or CSAT, but deep down, you know the experience with your brand could be better, you could have a sampling bias problem.

A few ways to avoid this problem is to:

• Regularly review your VoC program to identify and address any biases or shortcomings. This may involve adding or modifying survey distribution channels, timing, or frequency to ensure comprehensive coverage and minimize potential biases.
• Another great way is to analyze feedback data at a granular level instead of focusing solely on aggregated scores.
• Also, pay attention to qualitative comments (open ended responses) to gain deeper context into customer perceptions and experiences.

By doing these, you can mitigate the effects of sampling bias and gain a more accurate, actionable understanding of your customer's experience with your brand. This holistic understanding is critical for business improvement and sustained success.

In conclusion, I leave you with this question:

Are your customers truly satisfied, or are you just seeing what you want to see?