How we adapted eBPF for cloud-native telecom networks

Accurate anomaly detection and security monitoring are hallmarks of network resilience and operational efficiency. They are a major focus area for every telco but despite decades of experience and proven practices, have only grown more challenging due to complex, cloud-native architecture and increasingly sophisticated cyber threats.?

Operations teams are often overwhelmed with unnecessary alerts that result in inefficiencies and response delays. Static, rules-based systems are partly to blame due to lack of adaptability to changing network conditions and zero-day exploits. A finger can also be pointed at systems that rely too much on manual interventions, usually increasing costs and resolution times.?

With modern mobile networks becoming more complicated, relief may have arrived just in time in the form of a familiar technology.?

Extended Berkeley Packet Filter (eBPF) technology, which has long offered real-time, kernel-level visibility in cloud and enterprise networks, has demonstrated promising results for tackling anomaly detection and security monitoring in telecom networks. ??

We have witnessed this firsthand in the Rakuten Mobile network, where we have implemented eBPF for various purposes, including active measurements in transport layers, packet extraction from virtual network interfaces and the detection of unusual behavior. This has enabled us to identify performance issues or abnormal traffic in the transport network layers, address control plane issues, and respond to incidents more effectively in real time.

Additionally, we leverage eBPF for advanced intrusion detection and prevention, such as monitoring file system activities, restricting unauthorized access or modification of sensitive data, and detecting and blocking attempts to exploit kernel vulnerabilities. By combining eBPF with the latest statistical methods and AI advancements, we proactively detect performance deficiencies, identify attack vectors in clusters and mitigate risks while minimizing false positives.?

We believe this success is repeatable by other operators. Let’s dive into the results we have seen and consider a roadmap for any telco that wants to modernize network monitoring and anomaly detection with minimal performance impacts.?

A new approach for networks fraught with false positives

Telecom networks generate vast amounts of data, leading to thousands of alerts daily from traditional systems that rely on static thresholds and rules. Precision rates are a challenge, with many false positives caused by static thresholds, lack of context and limited use of machine learning. This overwhelms operations teams and creates alert fatigue. Resources end up being wasted on non-critical event investigation while over time, there is a loss of confidence and trust in the systems. This really becomes a problem when teams start to ignore repeated alerts, causing them to miss genuine issues.??

Legacy tech exacerbates this dynamic by relying on static thresholds and rules that are no match for the nuances and high variability of increased network complexity or emerging security threats like zero-day exploits that require adaptive methods to discover versus simply identifying anomalies that don’t match predefined signatures. It is also largely incompatible with cloud-native networks, struggling to effectively monitor distributed, virtualized and containerized telco environments.?

With telcos moving to more virtualized and containerized architectures, staying with legacy methods leaves operators at a big disadvantage.?

eBPF and AI for cloud-native telco networks?

eBPF was originally designed to provide real-time, kernel level visibility for cloud and enterprise networks. Think of network packets like a car driving down the road and eBPF as an agent that can instrument the engine of the vehicle or adjust its steering wheel while the car is still in motion. Critical adjustments and task monitoring can be carried out in real time.?

For this reason, eBPF has become an invaluable tool for monitoring, analyzing and responding to system events directly within the kernel without all the overhead of older user-space processing.?

While eBPF has a proven track record for cloud observability, networking and security in enterprise domains, it has rarely been applied to mobile networks. eBPF was not conceived with intention of deploying in telecom networks, which introduce complexity with unique challenges like dynamic traffic patterns and transport layer complexities.?

But combining eBPF’s capabilities with the latest AI advancements significantly expands the range of use cases it supports, providing quality and security assurance means with powerful features such as dynamic pattern recognition and context-aware anomaly detection.?

This opens the door to completely new approaches for detecting anomalies and threats. Static thresholds can be replaced with adaptive, real-time intelligence. Zero-day exploits that traditional systems usually miss can be more easily identified. Actionable insights can be provided to mitigate threats.?

Together, eBPF and AI can power real time threat detection, lower false positives by reducing noise and learning patterns, reduce performance impacts of monitoring and assume a more proactive security posture.?

In Rakuten Mobile’s experience, false positive rates were significantly reduced and the volume of unnecessary alerts decreased substantially. By incorporating AI, the accuracy of anomaly detection improved considerably compared to traditional methods.?

Adapting eBPF for the nuances of telecom networks

Mobile networks comprise complex, latency-sensitive and jitter-prone transport network layers that are usually monitored via passive systems that struggle to account for dynamic routing or rapidly changing transport paths.?

We set out to overcome these inherent limitations, understanding that mobile networks are prone to highly variable traffic patterns, including sudden surges and geographically dispersed flows, which eBPF was not originally designed to monitor given its focus on more predictable enterprise environments.?

Integrating eBPF into Rakuten Mobile’s distributed, containerized network architecture would require ensuring compatibility with virtualized and multi-tenant setups while avoiding performance impacts. Following are some of the key steps we took in the process:?

• Creating custom eBPF agents for transport layers. Rakuten Mobile developed in-house eBPF agents customized for mobile-centric metrics like transport network anomalies. These agents were tuned to aggregate and analyze data in real time to ensure zero delay detecting performance degradation.
• Integrating AI and ML for new capabilities. AI and ML algorithms like DBSCAN, in addition to advanced statistical methods, were adapted to account for telecom-specific needs. We tuned cluster techniques to dynamic traffic behaviors and created custom models to train Rakuten Mobile’s domain-specific data for better accuracy. Variational Autoencoders (VAEs) added sequential analysis capabilities for anomaly detection in time-series data.
• Minimizing impact on performance. eBPF’s lightweight design is meant to collect KPIs without overloading network resources. By performing data aggregation on each device where the eBPF agents are deployed, we reduced the volume of metrics required for processing, resulting in improved efficiency.
• Deploying iteratively. We started this journey with focused use cases like transport network anomaly detection to refine probes and algorithms before scaling more broadly. Continuous feedback loops helped us rapidly optimize integrations.
• Incorporating domain-specific insights. We leveraged our deep mobile network architecture expertise to fine-tune eBPF performance, tailoring thresholds to specific mobile conditions and adjusting models to detect telco-specific workload anomalies.?

Real-world results reveal what’s possible beyond our network

Once we successfully adapted eBPF’s capabilities to the telco network and overcame the myriad challenges posed by this unique architecture, the results were astounding.

By significantly reducing false positives, our operations team was able to focus more effectively on genuine issues. The improved detection accuracy of anomalies in network transport layers with hop-by-hop visibility, achieved through advanced systems compared to legacy approaches, minimized the need for manual interventions—saving time and reducing operational costs. Additionally, we enhanced our ability to detect and respond to zero-day exploits and other threats in real-time, strengthening overall network security.

Because there was overall less noise generated, teams could more quickly focus attention on critical issues with an improved ability to identify and address transport-layer anomalies before they impacted QoS.

We believe these results are all repeatable in telco networks around the globe for any operator adopting cloud-native architecture and technologies.

Advanced AI and ML models will continue to build on eBPF’s foundation, eventually enabling even more sophisticated anomaly detection. Those that successfully adopt this approach will be better positioned to handle the more stringent demands of 5G and future 6G networks.

David Soldani is SVP of Next Generation Advanced Research for Rakuten Mobile. Mention him in the comments to start a conversation and learn more.