5G & Multi Edge Application Delivery Networking - Role of Akraino/ICN & EMCO

Background

Content delivery networking is well known in the Industry, but it is limited to distribution of static content from several locations to the end users. Goal of CDN is to ensure high availability of content and faster access to the content by end users. Content includes static web text, graphics, scripts, documents, media files and downloadable software etc..

Many started to realize the content distribution is not good enough for some applications that generate content dynamically. For these types of applications, it is best if part of the application itself runs in geo-distributed fashion. By doing so, content can be generated locally and serve the end users and provide good user experience.

5G is increasingly being made available to end customers by operators. We all know that 5G provides higher bandwidth, lower latency and deterministic performance over 4G. Due to this, many think that a significant portion of access to applications and content would be over 5G.

In 5G, the component that bridges access networks to the DNNs (Data Network) is UPF. Unlike 4G, UPF specifications are designed to make this highly distributed. That is, UPFs can be deployed across multiple locations, closer to the Cell tracking areas. That gives an opportunity to place content and applications near the UPF and thereby addressing user experience needs of applications that generate content dynamically.

Pictorial representation of 'Application Delivery Networking' in edge world would look like this:

Questions to be addressed to realize Multi Edge ADN

1. Does the platform provide a way to place a copy of my cloud application (or part of the application) in right edge locations based on latency based on the location of my users?
2. What capabilities does my platform provide for me on when to choose to deploy my cloud application in a given edge location? Also when to delete the application from a given edge location?
3. Does the platform provide a way to redirect the traffic to a local copy of my application even if the user sessions are pointing to the cloud application? That is, does the platform provide a way to do this with no changes to any of my existing networking (DNS Servers etc..)
4. Does the platform provide a way to access cloud counterparts from my local copy of applications in both directions?

Current Akraino/ICN and EMCO (21.09 release)

Akraino/ICN consists of a set of extensions on top of traditional K8s to address Edge/Telco related gaps. EMCO is meant to deploy workloads across multiple edge locations, connect the applications together and perform life cycle management of workloads.

This combination can be a multi-edge application delivery platform. What additional features are needed in this platform to address questions listed above?

New features needed

Latency aware placement controller in EMCO: This is useful when administrators know where their users are already. This placement controller is expected to take input from the user on the latency requirements as intent for the applications and also take input on 5G tracking areas of these users. With the help of Telco APIs (5GFF), it can find out the right edge locations to place workloads.

Traffic redirection Action controller: This controller takes the intent from the administrator on which application microservices for which local traffic redirection to happen. As part of that, it can take input such as FQDN of each cloud application microservice that needs to be replicated in edges, and understand the K8s service name on which the microservices would be listening on. During instantiation of these microservices, this controller will create traffic redirection rules with the help of 'traffic redirection CRD controller' in the clusters on which these microservices are being deployed on.

Traffic redirection CRD Controller: This controller sitting at each K8s cluster will take instructions from 'traffic redirection action controller' of EMCO and automates the traffic redirection rules in UPF POD network namespace. Before doing so, it is expected to resolve the FQDNs to a set of IPv4 & IPv6 addresses and populate the rules in IPTables of UPF POD network namespace. Since, there is a not insignificant chances of IP addresses changing for FQDNs, it is necessary that this CRD controller also resolves IP addresses periodically or upon expiry of DNS records and populate IPTables rules with new IP addresses.

Demand Observer & Notifier: Application administrators may not deploy application instances proactively in some Edge locations. In those cases, Admin might want to monitor whether there are any users accessing their cloud application. Intention is that if users near that edge are using their application more frequently, the admin might choose to deploy the copy of the application in that edge. In the same way, if there are no users at the edge, he/she might like to remove the application instance in that edge.

This CRD controller can be used to monitor a specific application traffic by installing a CR. The CR needs to contain information such as FQDN of the cloud application instance, time quantity to update the status with statistics information. Say, if CR contains service1.example.com and time quantity as 1 hour, then this CRD controller monitors flows going to this service for each hour and puts comprehensive statistics (number of flows, total packets, total bytes) in the CR status. It may collect the last X (part of the CR) hours of status and then recycle the oldest one for a new hour.

Demand Monitor and Policy Manager: This module in EMCO is expected to do multiple things - To create Demand Observer CRs, to monitor the status of CRs to check whether it meets the requirements to bring up or bring down the application instance in that edge and to inform the orchestrator to bring up or bring down the application instance in that edge. Essentially, it takes input as intents and uses those intents to create demand observer CRs and to verify CR status to make decisions to bring up/down the application instance.

Considerations

Few must be considered in developing Multi Edge ADN.

• Instantiation of application in the same K8s cluster as UPF.
• Instantiation of application in different K8s clusters than the UPF.
• Application instantiation exposed directly without any ingress proxy.
• Application instantiation exposed behind service mesh such as ISTIO/Envoy.
• Multiple UPF instances in a slice in a K8s cluster
• Network Slice support where there is one set of UPF in each slice in one K8s cluster.
• Don't expect the applications are configured with right routing entries to send the reverse traffic via UPF and more over the same UPF instance. SNAT may also need to be adopted to ensure that the traffic goes via the same UPF instance.

Any design of CRD controller, EMCO controllers, sidecars shall consider the above. Note that the method described above is assuming that Linux Kernel is used by UPF for traffic redirection and routing. An implementation of 5GC by free5GC adopts Linux kernel and hence this method is expected to be tested with free5GC. UPF implementations that are VPP/DPDK based would require additional enhancements to this method and is currently out of scope for Akraino/ICN.

Summary

Many edge computing deployments that are envisaged mostly depend on the DNS Server configuration. That is, each application instance deployed in each edge will have an entry in the global DNS Servers such as Route53. Here, the onus on the selecting the right application instance by UE application is with the DNS Server.

In the case of 5G, there is another method where edge copies of applications can be deployed where the UEs get the best possible experience without changing any network configuration such as DNS Servers, routing entries at the applications. Also, this method provides intelligence on when to bring up/down the application instances in the Edges with more accuracy. This post described this method and what is involved in realizing this method in two open source projects Akraino/ICN and EMCO.