Kubernetes and TCP/IP Networking

By: Raj Sahu

Kubernetes (k8s) is undoubtedly the most decorated cloud infrastructure. But, how can Networking/or Network Engineers contribute to it? Kubernetes can be deployedon on-premise infrastructure or Public Cloud managed services, such as GKE [for GCP], EKS or Fargate [for AWS]. Since I got to work on both side of the houses namely Network and Cloud, I am trying to share my learnings - a tiny attempt, to decipher cloud infrastructure in order to encourage, onboard Network engineers Cloud journey. Hope as you follow along, you will be able to correlate, appreciate and also extrapolate networking aspects of Kubernetes and most important establish relevancy of networking skills in managing scaled Kubernetes Clusters.

For example, Supported scale for GKE versions 1.18, Nodes per cluster:15,000?, Nodes per node pool zone:1,000, Pods per node:110, Pods per cluster:150,000, Containers per cluster:300,000

In this blog, I will only try to cover Networking aspects and Attack Vectors, which in my opinionise very relevant to Network Engineers. For Kubernetes deep dives I would like to recommend, Nana's Kubernetes Youtube Channel , Kubernetes Deep Dive - Nigel Poulton & Kubernetes Security - John David Marx Acloudguru Courses.

We are aware of Monolithic services as shown in figure 1, Where single server could terminate user API calls, Authenticate users, serve requests.Monolithic architecture is a single-tiered application where all the different factors from a single platform can be united and used for a single program!?

The microservice architecture replaced Monolithic applications as it enabled the rapid, frequent and reliable delivery of large, complex applications. It also enables an organization to evolve its technology stack Asymmetrically.

Above microservices functional and scale requirements are mostly served by Kubernetes . Each Microservice can be effectively run on container, a container is packaged inside pod. Figure 3 shows components which go in K8s cluster

Before we go further along, let me summarize Networking functionality in Kubernetes broadly address below problems:

• How cross-node pod-to-pod connectivity (for east-west traffic) is achieved.
• How the services running in the pods are discovered by other pods, and how pod-to-pod traffic is load balanced when consuming a service.
• How services running in the pod’s are exposed for external access from clients outside the cluster (for north-south traffic).
• How with network segmentation, pods are secured by restricting network access to pods
• How high availability, global load balancing etc can be achieved in federated multi-cluster deployments

As microservices scales and communiate with each other, Kubernetes allows all nodes to communicate with each others [via TLS API for North-South(endpoint access from outside cluster to API server) and East-West (control plane traffic for example between API server and Kublet or between scheduler and API server or any other cluster components) ] and also all pods can talk to each other without NAT (Network Address Translation). Each pod gets its own IP address. Now it’s evident that under the hood Kubernetes has 3 separate networks - Pod Network, Node Network, Service Network

As shown in Figure 5, pod to service translation is done by kube-proxy packaged within pods using iptables

Figure 6 illustrates end to end IP communication between pod to pod.

.

Kubernetes Container Security - Now that we know how all nodes can communicate with each other without any restrictions. A single node if compromised (despite TLS APIs) can pose security restrictions to entire Kubernetes cluster. Kubernetes Security is a specialized field in itself. ?For K8s security specific use case we have multiple solutios, adding 2 which I havesome bexposure to Project Calico and Cisco-ACI solution

As Network Engineers, we have 3 separate networks to manage and trouble shoot for a secure inter-pod or inter-node communication. How can we troubleshoot with tcpdump, for that we have yet another local network within the pod as shown in Figure 7,called Side Car,a side car containerh share disk and network with application container and hence can capture network traffic via tcpdump. Interesting read Using sidecars to analyze and debug network traffic in OpenShift and Kubernetes pods

Securing Kubernetes against Attack Vectors - yet another Novel Networking Problem Statement

Now that we are aware of networking aspect of Kubernetes, I would like to shed some light on Attack vectors and how Network Engineers can help secure Kubernetes Cluster.

An attack vector [as shown in Figure 8] is a path or means by which a hacker can gain access to a computer or network server in order to deliver a payload or malicious outcome. utcome. Attack vectors enable hackers to exploit system vulnerabilities. Figure 8 points out possible 8 such Attach Vectors

Since Kubernetes nodes are bare-metal or virtual hosts, hackers may use point of entry to the host such as ports open for shell access to gain control of the system.

Once inside, the attacker may inject malicious code and other vulnerabilities that would be used to further compromise the system and others on the network, as well as siphone off data and other cyber assets.

The advent of containers and Kubernetes hasn’t changed the security mission and networking aspects. Goal is still to make it difficult for bad actors to break into your applications and its infrastructure.?Securing against ManyAttack Vectors described above and managing effieciently internal networks, definitely creates challenges but also opportunities for network engineers to innovate, secure and manage highly scaled Kubernetes Clusters

Happy Cloud Learnings!!!

#AWS #GCP #GKE #vulnerabilities