Navigating the Cloud: Demystifying Amazon VPC CNI for EKS Networking

In this blog we will cover some of the most important concept relate with EKS cluster and why its a special one which has been widely used by enterprises now a days .

Topics in this Blog :-

1. What is AWS CNI ?
2. Why AWS CNI ?
3. Benefits of CNI
4. What is Network namespaces and how it's relate with CNI ?
5. Steps to install CNI in EKS .

What is AWS CNI ?

AWS CNI is the default plugin or an add-ons while we install EKS cluster through cluster or via Terraform . This CNI ensures complete networking communication between pods & containers .

Let's deep dive step by step .

Why AWS CNI ?

In typical enterprise where we have couple of Clusters spans across multiple AZ and they have certain Nodes and underneath pods exists .

Let's take this scenario as follows :-

a. We have multiple EKS clusters spanning across multiple AZ's for the sake of HA , DR purpose .

b. Each cluster having 5 nodes with 20 pods .

c. I have to ensure pods across nodes can communicate using namespaces , label in each AZ's .

d. In peak traffic scale the nodes up to Desired :- 7, Min :- 5 Max :-10

Now how the CNI will assist us to ensure proper networking between pods across nodes and also between nodes in a cluster within an AZ ?

1. IP Prefix Delegation: Imagine you have a big neighborhood (your VPC) with many houses (nodes). Each house has a limited number of house numbers (IP addresses).IP prefix delegation is like getting a whole street (prefix) of house numbers for each house.
2. How It Helps Your EKS Clusters: You have multiple neighborhoods (EKS clusters) across different areas (AZs). Each neighborhood has 5 houses (nodes), and each house hosts 20 people (Pods). IP prefix delegation ensures that each house gets a whole street of house numbers (IP addresses) for its people (Pods).
3. Benefits: With a whole street, each house can assign addresses to more people (Pods). People (Pods) in different houses (nodes) can talk directly using their house numbers (IP addresses). When more people move in (scaling nodes up), each house still has enough addresses for everyone.

Benefits of CNI

• Pod IP Address Assignment :- The VPC CNI plugin assigns a private IPv4 or IPv6 address from your VPC to each Pod. This ensures that Pods have consistent IP addresses within the VPC network. Predictable IP addresses simplify network management and security policies.
• Same IP Address Space as VPC: Pods share the same IP address space as other AWS resources in your VPC. This seamless integration allows direct communication with other services (e.g., RDS, S3, Lambda) without NAT gateways or additional configurations.
• VPC Flow Logs and Security Groups :-VPC CNI enables flow logs for network traffic between EC2 worker nodes and Pods. Security groups work seamlessly, allowing you to assign policies directly to Pods. Enhanced visibility and security controls are crucial for enterprise workloads.
• Efficient Network Communication :-Containers within the same Pod communicate using local ports . Reduced overhead and latency improve application performance.

What is Network namespaces and how it's relate with CNI ?

• Imagine you have a big apartment building with many floors. Each floor is like a separate world with its own rooms, doors, and rules. Network namespaces are like these separate floors, isolating network resources for different purposes.
• Now, let’s say you have two apartments (containers) on different floors (namespaces).
• Containers in different namespaces can’t directly see each other’s doors or rooms.
• The Container Network Interface (CNI) acts like a bridge between these floors, allowing communication between containers across namespaces.

How They Relate to CNI:

• Example:

Imagine Apartment A (Namespace A) and Apartment B (Namespace B). Apartment A has a kitchen (container A1) and a bedroom (container A2).Apartment B has a living room (container B1) and a bathroom (container B2) . CNI sets up a hallway (virtual network) connecting these apartments: Containers A1 and B1 can chat in the hallway (cross-namespace communication).Containers A2 and B2 can’t directly see each other but can use the hallway to talk .

How shall I ensure that CNI installed in my EKS cluster ?

To ensure that the Amazon VPC Container Network Interface (VPC CNI) plugin is installed in your Amazon EKS cluster, follow these steps:

1. Check the Amazon EKS Add-On Type:

kubectl get addon  -A        

• Look for the vpc-cni add-on. If it’s listed, the VPC CNI plugin is installed.

Verify the Environment Variable:

• Describe the aws-node Pod in the kube-system namespace .

kubectl describe pod aws-node-<random-string> -n kube-system        

• Look for the environment variable AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG.
• If it’s set to true, custom networking (VPC CNI) is active.
• Verify that the Elastic Network Interfaces (ENIs) are configured properly.

kubectl get ENIConfig -A -o yaml        

• Ensure that the ENIs are associated with the correct subnets and security groups.

Remember to replace <random-string> with the actual Pod name.

In conclusion AWS CNI offers rich and powerful features to manage network and its traffic across pods within nodes and across nodes to scale , deploy and expose our application .

Keep Exploring , Keep Learning new things .