SERVICE QUALITY OVER SLAs and the WATERMELON EFFECT

At one point we have been in a review meeting where a service provider proudly stated that all SLA’s were met with the five or seven nines metric, or more still your service manager has given you praise for meeting your SLAs targets. But does this always translate that the user/customer is satisfied? Probably NOT.

It is the responsibility of a service provider to meet the SLA targets of its customer, NOT just for the sake but also to ensure that the customer is contented with the quality of service provided. However, there are customers that are not satisfied with the outcomes delivered yet the service provider claims to have fulfilled the SLA. Such kind of scenarios give birth to watermelon SLAs where its green on the outside (targets met), but red on the inside (customer is not satisfied).

The support provider sees green, while the customer sees red.

Such scenarios are common in IT service-level agreements (SLAs) where the SLAs don’t reflect the real service given, nor the service experienced by users. This is because the service provider wants to achieve the SLAs on time at the expense of the customers’ contentedness. This thirst of achieving SLAs before they turn red, has created a discrepancy between reports and reality creating a scenario whereby Key Performance Indicators (KPIs) are often developed without proper consideration and understanding of the reasons behind the measure.

Service managers should not concentrate on designing and reporting on metrics that have little value and do not demonstrate the contribution to an organization’s business outcomes. There is more to service management than just reports or metrics therefore making it important to ask oneself why we choose what we are measuring and the purpose behind measuring that metric.

When we focus on numbers, all we see are the numbers, not the end-user experience

The SLA for a service may have an availability target of 99.5% meaning if the service provider hits that – all is good. Right? No – not if that 0.5% of downtime hits when that specific service is NOT available. It instead needs to be all about the end-user experience. There is need to shift the paradigm from measurements and metrics to business impact. SLA targets may well have been met but do the users feel good about using that particular service when they most need it?

Five nines mean that systems and services are available 99.999% of the time. It also means that both planned and unplanned downtime is less than 5.26 minutes per year.

99% - 87 hours 36minutes

99.5% - 43 hours 48min

99.95% - 4 hours 23min

99.99% - 53min

99.999% - 5min

When a measure is a target, it ceases to be a good measure – Goodharts Law

WAY FORWARD?

A possible workaround for this is to also include the Customer Satisfaction (CSAT) as a comparative measure. This way transactional satisfaction/dissatisfaction can be identified and when aligned to SLA performance can help tell a more complete story around the delivery for a measured period.

There is nothing worse than walking into a client meeting thinking everything is OK as the metrics are green, then the meeting turns into a complaint session. Ultimately, if it is Customer Satisfaction that is sought, then this should be measured alongside process-oriented metrics. CSAT is definitely a good sense-check but it’s also often not good enough, in terms of granularity and focus, to spot the hidden expectations gap. Measuring employee experience, can highlight a number of issues with IT support in particular.

SUMMARY

It’s not uncommon for SLA reporting to be a sea of green targets, while the business perception is not nearly as great. While the “if you don’t measure it, then you can’t manage it” mantra rings true much of the time, I prefer “if you measure the wrong things, then you’ll get better at the wrong things” quote. Ultimately, if the business is unhappy with service provider, then the service provider needs to have honest conversations with stakeholders to find out what they value, and which metrics will report on the things that really matter to them.