Ditching the "Averages"

"Whenever I read statistical reports, I try to imagine my unfortunate contemporary, the Average Person, who, according to these reports, has 0.66 children, 0.032 cars, and 0.046 TVs." - Kato Lomb

Well jokes apart.....for anyone who has spent a considerable amount of time tweaking their customer’s experience with the product or with the post-sales Support, knows this fact already that average indicators of key performance metrics are anything but a complete indicator of the true experience that your customers are having.

For keeping things in context, in this article, I would be referring to the use of percentiles, only for the purpose of driving performance of Customer / Technical Support centers.

While the effectiveness of percentiles is theoretically sound and used by many engineering teams to gauge and tweak the performance of their product, many product support centers continue to use averages for performance reporting and driving improvements, as they are easy to report and can be implemented across all tools available out there. 

Common reasons why averages are common, percentiles are not

• Difficult to implement

This is by far the most common reason I have come across while interacting with peers. A lot of the CRM / ticketing tools out there, do not support percentiles in their vanilla offerings, thereby making it difficult to use.

• Easily Measurability of Averages

Averages are everywhere. You don’t need to go find them. True in life and true for metrics as well.

• Averages are comfortable

It is way too easy to sandbag your way around with averages. All you need is a majority of tickets well below the thresholds and all your sins are forgotten. Literally!

Percentiles vs Averages : The BIG Difference

Simply put Percentiles and Averages are just two perspectives of looking at the same scenario. However, this seemingly small change of perspective makes a world of difference. Let’s take an example:

In the above example if we compare the Avg. First Response Time (FRT) for May 2017 and Aug 2017, there is not much of a difference and the variation in performance would typically be passed off as “within tolerable limits”.

For the same months, when we shift our perspective towards the 90th percentile for FRT we find that 90% of tickets in Aug 2017 were responded within 1.2 hrs or 72 minutes while in May 2017 90% of tickets were responded to within 2.1hrs or 126 minutes. This is a huge difference. In some cases, it could be the difference between a delighted customer and a not so pleased customer.

Therefore, we find that averages only end up giving us the “Beer Goggles” effect while percentiles show us the true north. Of course, measuring the percentiles for just one metric on your dashboard would not help drive your customer’s experience towards awesome. You would need to do this for most metrics on the dashboard. While there are some metrics like the First Call Resolution (FCR) which cannot be subject to percentiles, most other important metrics can be measured as percentiles, provided of course, your analytics tool /CRM provides you this functionality.

==============================================================

P.S:

1.) All of the data points in the above article are random examples and do not represent or resemble actual data or trends from any of my past or present employers.

2.) The above article is simply a gist of my observations gathered during the course of my career and my only intention is to share ideas that have worked wonderfully for me. All of this with a humble intention to make this world a better place by delivering and helping deliver an amazing customer experience!

Views and comments are more than welcome!