Quality of End-User Experience in Mobile Networks – How to Measure the Ground Truth?

Mobile phone data usage has shifted from internet browsers to apps. Virtually all internet services are available for mobile users as dedicated mobile apps. Factors impacting the quality of end-user experience (QoE) in mobile networks include?the mobile device, radio network, core network, possible traffic shaping policies applied by the service provider, and application servers; and the mobile apps are adding another variable into the mix. We have listed here the top three reasons why active testing with real applications is the smartest way to go when measuring end-user QoE.

1. Mobile networks adapt to applications

A key consideration in QoE and Quality of Service (QoS) measurements is that the network traffic flow in a live network can be optimized based on the type of traffic, including specific applications or application types. Traffic shaping is a QoS and network congestion management method that controls network data flows by allowing higher-priority traffic to proceed at optimal levels even when the network becomes overutilized and by restricting certain applications to reserve network capacity for other users. Traffic shaping can be used to optimize network performance by prioritizing certain traffic flows and ensuring an individual user’s traffic rate does not exceed the bandwidth limit. Network resources can be allocated to individual application sessions adaptively and dynamically.

The impact of traffic shaping and other application specific network adaptations can be included in QoE assessment only when testing with real apps against real app servers. The traditional way of using a simple bulk data transfer or synthetic traffic generated with a purpose-built test protocol in the testing are missing this important aspect of end-to-end QoE.

2.??????Many applications adapt to the mobile network

Not only does the network adapt to applications, but also many advanced applications are adaptive and try to cope with network bandwidth changes dynamically to guarantee service to their users. For example, Microsoft Teams can adapt to bandwidth-limited network conditions by changing the audio and video codec bitrates. Such network-aware, dynamic behavior is very difficult to mimic with synthetic traffic active testing.?

3.??????Traffic emulation may not always trigger adaptive behavior

Synthetic traffic-based active tests are widely used in the industry for application QoE testing. TWAMP and equivalent protocols provide a way to generate different traffic profiles for emulating real end-user data traffic. The synthetic testing method has its place in machine-to-machine and industrial use cases and with some services where active testing with real app is impractical. However, measuring QoE with emulated traffic may not always trigger traffic shaping in the same way as real consumer applications will. Furthermore, the advanced adaptive behavior of the latest native applications is difficult to mimic with traffic profiles.

Keysight provides two methods for measuring with real native applications

The first approach to measure with real applications is to use the application programming interface (API) to understand the interaction of the application with the network by controlling and monitoring the real application. The benefit of this method is that via API, several status and process indicators from the IP and application can be collected. On the other hand, the limitation of this approach is that a stable API is available only for a limited set of applications, such as, WhatsApp, Line, Viber, BIP, and Instagram.

Application Testing Automation (ATA) with Nemo Outdoor

The second approach, Application Testing Automation (ATA) method, was recently introduced by?Keysight. This unique artificial intelligence (AI) and machine learning (ML) based method controls and monitors applications through the smartphone screen, in the very same way as real users use the application. With ATA, it is possible to quickly and accurately assess users’ interaction with the real mobile application, giving realistic insight of the network performance even in the most complex and dynamic circumstances. The benefit of this approach is that it enables automated testing of most applications in the market and there is no dependency on APIs from application vendors.

Testing Automation supports testing with some of the most popular applications, such as Facebook Messenger, Microsoft Teams, Snapchat, TikTok, and Zoom. The new ATA testing method is available with Keysight’s?Nemo Outdoor?based 5G drive test solutions.?

Contact us for more information: [email protected]