About Quality of experience (#oldie)

(Originally posted in NSN NPO User Group, November 2010.  Interesting to look back every now and then and check your old points of view.)

It is war.

That simple. CSP’s Executive Boards are increasingly becoming the battlefield where CTOs and CMOs exchange head-ripping blows about one of the sacred topics of the techies:

Network Performance.

Indeed, it is a bloody war where no ammo is spared: Alarm Statistics vs. Customer Complaints, KPIs vs. Churn figures, and the reason for its bloodiness is that the figures don’t match: KPIs look better than ever, alarms are at an all-time low but customer complaints soar while churn sits on the far side of 3%.

Monthly.

Put into perspective, European retail banks have an average churn rate of 7%. Yearly.

“Quality of Experience”, or its infamous acronym –QoE– is slowly but surely making its way from the CSP’s shiny Marketing offices deep down into the O&M underworld. It is the magnitude to measure the subjective performance of the network services, or put into other words, customers’ perception.

How do we measure subjective? We lack a Subjective Performance Management system (SPMS) that delivers Key Subjective Performance Indicators (KSPI), or at least we know we cannot buy one.

Personal Communications has been a KPI-locked sector for so many years that maybe we will better describe QoE by listing what it is not.

• It is not a KPI. If it was, wouldn’t it have been found yet? 
• It is not Quality of Service (QoS), though it is related to it. 
• It is not an exclusive issue of your already stressed Radio Optimization team. 
• It is not constant across segments, or time, or space, or… It is intrinsically linked to your customer. 

How to measure QoE and, yes, how to improve it.

In our opinion, a good metric of QoE must be linked to Service Performance and aggregate relevant inputs from miscellaneous sources: network elements, interface traces, application servers, terminals, agents…

When is an input relevant? When it refers to the service in question. This equals saying that there will not be one QoE, but as many as services: Browsing QoE, MMS QoE, Voice QoE,… It is difficult to say this is not correct.

One can also expect several categories of metrics.

• Key Quality Indicators (KQIs), which are closely linked to the service in question and which can be for example Retainabilty, Accesibility, Integrity. 
• KQIs are made of Key Performance Indicators (KPIs), which relate to the Network Element, interface or application. The stuff techies know best. 
• Counters, Clear Codes, Alarms… are the most detailed metrics which build the KPIs. 

Besides metrics connected to QoS, the QoE model needs thresholds. We need to know whether the obtained values are good enough, so it can be displayed clearly.

In the end, it all boils down to making changes that improve the customer perception. So the model needs to allow drill-down from KQI to counter in such a way that it can be possible to explain the overall QoE value by observing its different parts. On every step, every magnitude will be broken down to its components and each of them will require a threshold.

The thresholds will demand validation. Perceptive Solutions such as Mobile Quality Analyzer can provide the necessary correlation between QoE and QoS.

It is important to note that metrics and thresholds will not be static: new services or new sources of information will affect the metrics, whereas the thresholds must adapt to the customers’ expectations. So, the QoE model needs to be in permanent adaptation.