EP 11 - Network Policies and Service Mesh: Which One Do You Need?

Networking in Kubernetes

Networking in all areas of Infrastructure Engineering (DevOps, SRE, Platform Engineering) is more crucial than most people assume. It is the backbone of everything. And in the world of Kubernetes, where multiple services need to talk to each other, it is more important than ever.

Kubernetes is created to foster micro-service computing. To do that, various services or applications running in pods need to be able to send packets of data to each other, hence the need for managing and securing networks.

Controlling and Managing Network Traffic

By default, Kubernetes has an “any-any” network traffic policy. This means that the default configuration of Kubernetes allows network traffic from any pod within the cluster to reach any other pod within the cluster. This might not sound like an issue at first because it is a good thing for pods to be reachable within the cluster, however, when you begin to work in an environment that needs strict privacy within namespaces or for certain pods, it becomes an overly permissive policy.

For instance, I have within my Kubernetes cluster production databases, Kafka brokers, and cost monitoring services that I do not want to be accessible by every pod or user who has access to my cluster. Hence, the default networking policy of my Kubernetes cluster is now a security and compliance concern for me. What do I do about this?

There are two main tools we can use to manage the traffic within our cluster, and today we will look at both of them, their feature, benefits, and advantages and try to determine which solution is best for you to implement in your cluster.

They are:

• Network Policies
• Service Mesh

Network Policies

Network policies are a mechanism for controlling the pods and Kubernetes services. They comprise rules that either allow or deny traffic to and from a pod based on the result of evaluating certain criteria or conditions.

Network Policies function on the Network and Transport layers of the OSI model (Layer 3 and Layer 4), and as such, they control traffic using the IP address and port of the pods and also the namespaces.

Network policies are basic Kubernetes objects and come with the Kubernetes API. However, to implement network policies, you must have a CNI that supports the execution and enforcement of network policies. Kubernetes allows the specification of network policies but cannot implement them by default.

How it Works

1. First, you need a CNI that can implement network policies
2. Next, you need to define the network policies using components like pod selector, namespace selector, labels, ingress rules, egress rules
3. Apply the network policy
4. Confirm it works by testing it

Features

Some core features of Network Policies include:

1. Pod Isolation: Network policies help you control what traffic can reach a pod
2. Ingress and Egress Traffic Control: You can manage the traffic that is allowed in as well as traffic that goes out
3. Label-Based Selection: Network Policies are attached to pods using labels
4. Namespace Isolation: You can also attach policies to namespaces using the namespace selector
5. Default Policies: Network policies enable you to easily apply some common policies without explicitly writing rules. Default policies include: allow-all-egress, default-deny-all, allow-all-ingress, default-deny-egress

Pros and Cons

1. Pro: They are faster than Layer 7 policies (Service Mesh)
2. Pro: You can implement weighting
3. Con: They work only on Layers 3 and 4 and, as a result, cannot offer advanced features like authorisation and authentication

Service Mesh

Service Meshes, on the other hand, also provide a means for controlling and managing network traffic, but they do this on Layer 7 (The Application Layer), and because of this, they can have advanced features like advanced load-balancing, throttling, authentication and authorisation of requests etc. They control traffic based on application headers and paths.

How it Works

The main idea behind the operations of service meshes is to distribute a set of proxies, which would be embedded into the pod as sidecars and used to intercept traffic to the pod.

Process:

1. The app requests another application
2. Service mesh applies dynamic routing to determine the destination of the request
3. Forwards request to its destination based on a set of requests and constraints

Features

Some of the features that service meshes provide:

1. Traffic Management: Advanced routing, traffic splitting, and load balancing between services.
2. Service Discovery: Automatically detects and connects services within the mesh.
3. Security: Mutual TLS (mTLS) encryption for secure communication between services.
4. Observability: Metrics, logging, and distributed tracing for monitoring service interactions.
5. Resiliency: Circuit breaking, retries, and failover to handle failures gracefully.
6. Policy Enforcement: Access control, rate limiting, and security policies across services.
7. Load Balancing: Efficient distribution of traffic between service instances.
8. Authentication and Authorization: Verifies service identities before communication.
9. Zero-Trust Security: Implements security measures without needing to trust the internal network.

Pros and Cons

1. Pro: Service Mesh offers Advanced Security through encryption, authentication and other mechanisms
2. Pro: With logs metrics and tracing, Service Meshes offer Improved Observability
3. Con: Increased Complexity: Managing the control plane and data plane components of services meshes introduces complexities, engineering overhead and increased cost
4. Con: Resource Overhead: Service Meshes run as sidecars and, as such, consume CPU and memory resources
5. Con: Latency Issues: Proxies add extra hops in communication, potentially increasing request latency, though this can be minimised with proper configuration.

Differences

Which Do You Need In Your Cluster

The choice of whether to use Network Policies or Service Mesh all comes down to your specific use case. It is important, however, to avoid deploying a service mesh if you don’t need to to avoid introducing unnecessary complexity and latency.

Install Network Policies If:

1. Security is your primary concern, and you want to control which services/pods can communicate with each other.
2. You need basic traffic filtering at the network level to ensure that unauthorised traffic can’t reach a pod.
3. You want a lightweight solution that doesn’t introduce significant overhead.
4. You are not necessarily looking for advanced features like load balancing, retries, or observability, and only need simple network traffic rules (Layer 3/4).
5. You are focused on Kubernetes-native solutions and don’t want the complexity of additional infrastructure.

Install a Service Mesh If:

1. You need advanced traffic management features like load balancing, retries, circuit breaking, and canary deployments.
2. Observability is crucial, and you want to have detailed insight into service-to-service communication via tracing, metrics, and logging.
3. You want to enforce mutual TLS (mTLS) for end-to-end encryption and authentication between services.
4. You have a microservices architecture and need advanced features to manage inter-service communication at scale.
5. Resiliency and fault tolerance are critical for your system, and you want automatic retries, failover, and circuit breaking.
6. You are okay with the additional resource overhead from sidecar proxies and the operational complexity of managing a control plane.

In summary, choosing between Network Policies and a Service Mesh depends on your specific needs. For lightweight security and traffic control, Network Policies offer a simple, effective solution. However, if your microservices architecture requires advanced traffic management, observability, and security features, a Service Mesh is the right choice. Assess your cluster’s complexity and scale to determine which approach best fits your goals, and remember, combining both can provide robust control and flexibility for your Kubernetes environment.

Thanks for reading and Happy Engineering ????????