Kubernetes Networking

Kubernetes provides network model/principles and additional tools implement it. Network policies are implemented by the network plugin. To use network policies, a networking solution is to be used. This networking solution should support s network policy.

Plugins come in two options

1. CNI Plugins i.e. container network interface plugins: CNI plugins in Azure can be Azure CNI or Calico Plugin. Using Calico gives option to create network policies.
2. Kubenet plugin: Kubenet is a very basic, simple network plugin, on Linux only. It does not, of itself, implement more advanced features like cross-node networking or network policy. It is typically used together with a cloud provider that sets up routing rules for communication between nodes, or in single-node environments.

Popular Networking solutions/tools :

• Flannel
• Calico
• Weave
• VMWare NSX

FLANNEL Networking Concepts

Flannel is created by CoreOS for Kubernetes networking; it also can be used as a general software defined network solution for other purpose. To achieve Kubernetes' network requirements, flannel’s create another flat network i.e. overlay network which runs above the host network. All containers (Pod) are assigned an IP address in this overlay network, they communicate with each other by calling each other’s IP address directly.

Flannel solution created an overlay network on host network (from cloud VPC) which spans across the Kubernetes nodes a.k.a. k8s nodes. Each pod gets an IP from this overlay network. In Inter pod communication, say container-1 on node-1 needs to connect to container 2 on node 2, the packet goes via docker0 bridge which acts a container gateway. This packet destination address gets listed in node route table created by flannel daemon (flannel) which runs on each host or K8S node. As the destination IP is in overlay network range, it sends the packet to flannel0 which is a TUN created by Flanneld. Flannel stores the subnet to host mapping information into the etcd service, so each flanneld process queries etcd to know each subnet belongs to which host and compares the destination IP address with all subnets key stored in etcd. Accordingly, it sends the packet to node 2 and then it sends the packet to destination from eth0 i.e. node-2 nic to flannel0 to bridge0 network of node 2 and then to destination.

Calico Networking

Calico is an open source networking and network security solution for containers, virtual machines, and native host-based workloads. Calico supports a broad range of platforms including Kubernetes, OpenShift, Docker EE, OpenStack, and bare metal services. Calico comes with different networking options. However, Calico recommends running Calico without use of Overlay network / encapsulation. This provides highest performance and ease of network. Generally Calico is used mostly in cloud environment with CNI for Network policy enforcement.

Azure Kubernetes AKS Networking

1. To provide network connectivity, AKS clusters either use kubenet (basic networking) or Azure CNI (advanced networking).
2. By default, AKS clusters use Kubenet, and an Azure virtual network and subnet are created for you. With kubenet, nodes get an IP address from the Azure virtual network subnet. Pods receive an IP address from a logically different address space to the Azure virtual network subnet of the nodes. Network address translation (NAT) is then configured so that the pods can reach resources on the Azure virtual network.

With Azure Container Networking Interface (CNI), every pod gets an IP address from the subnet and can be accessed directly. This approach requires more planning, and often leads to IP address exhaustion or the need to rebuild clusters in a larger subnet as your application demands grow. Azure CNI is being used with Azure network policies or by Calico policies.