Kubernetes security 3/4 - DevOps engineers perspective

How Can Your Kubernetes (K8s) Deployment Be Best Secure?

Due to its complexity, Kubernetes requires a lot of configuration and administration. You must address critical architectural flaws and platform dependencies by following security best practices if you want to make Kubernetes workloads secure, especially in a production environment.

1. Enable Role-Based Access Control in Kubernetes (RBAC)

You may specify who has access to the Kubernetes API and what rights they have with the aid of RBAC. On Kubernetes 1.6 and later, RBAC is often turned on by default (later on some hosted Kubernetes providers). When you implement RBAC, you must also disable the old Attribute Based Access Control since Kubernetes mixes authorization controllers (ABAC).

Choose namespace-specific rights over cluster-wide permissions when using RBAC. Don't give cluster administrators access, not even while troubleshooting. It is safest to permit access only when essential for your particular circumstance.

2. Make Use of Third-Party Authentication for the API Server

Integration with a third-party authentication service provider for Kubernetes is advised (e.g. GitHub). This guarantees that the Kube-apiserver does not alter when users are added or removed and adds extra security features like multi-factor authentication. Make sure users are not managed at the API server level, if at all feasible. OAuth 2.0 connections like Dex are also an option.

3. Use a firewall, TLS, and encryption to protect etcd

etcd is a sensitive resource and a prime target for attackers since it keeps the cluster's state and its secrets. Unauthorized users who obtain access to etcd can take control of the whole cluster. Because bad people can use read access to increase their rights, it is also risky.

Use the following configuration parameters to set up TLS for etcd for client-server communication:

• the certificate needed to establish an SSL/TLS connection with etcd; the certificate key file (not encrypted) which is cert-file=
• —client-cert-auth: Indicate that etcd should search for a client certificate certified by a reputable CA in incoming HTTPS requests.
• Trusted certification authority —auto-tls: <path>
• —trusted-ca-file= For client connections, use a self-signed, automatically created certificate.

Use the following configuration choices to set up TLS for etcd for server-to-server communication:

• —peer-cert-file=: Certificate used for peer-to-peer SSL/TLS connections
• Certificate key: —peer-key-file which is the comment for not encrypted
• When this option is enabled, etcd examines all incoming peer requests for valid, signed client certificates.
• Trusted certification authority: —peer-trusted-ca-file

Use come Practices to define your needs on the cluster

1. Use automatically produced self-signed certificates for peer-to-peer connections with peer-auto-tls.
2. Additionally, install a firewall between the etcd cluster and the API server. Use Calico to set up a firewall on a different node that is running etcd, for instance.

For etcd secrets, enable encryption at rest:

Although it is not enabled by default, encryption is essential for protecting etcd. By supplying the parameter -encryption-provider-config to the Kube-apiserver process, you may activate it. You must declare your secret keys and choose a provider to handle encryption within the setup. For further information, consult the documentation.

4. Isolate the nodes of Kubernetes

Nodes for Kubernetes must be on a different network and shouldn't be directly accessible from open networks. Even better, stay away from direct connections to the main corporate network.

Only if control and data traffic in Kubernetes is segregated is this feasible. If not, both pass over the same pipe and unrestricted access to the control plane follows unrestricted access to the data plane. Nodes should ideally be set up with ingress controllers that are configured to only permit connections from the master node on the designated port through a network access control list (ACL).

5. Track Network Activity to Control Communications

Cluster networks are typically heavily utilized by containerized applications. To understand how your application interacts and spot unusual communications, monitor live network traffic and contrast it with the traffic permitted by Kubernetes network policy.

In addition, you may spot network rules that aren't being used by cluster workloads by comparing active traffic to authorized traffic. By eliminating unnecessary connections to lessen the attack surface, this information may be leveraged to improve the approved network policy even further.

6. Employ Process Whitelisting

The use of process whitelisting to identify unexpectedly running processes is highly effective. To start, keep track of all processes that are active when the program behaves normally over some time. Use this list as your whitelist moving forward to control future application behavior.

Runtime analysis at the process level is challenging. Several commercial security tools can evaluate and spot irregularities in processes operating across clusters.

7. Switch Audit Logging on.

Ensure that audit logging is enabled, and that you are keeping an eye out for any strange or undesirable API requests, particularly failed authentications. A "Forbidden" status notice appears next to certain log entries. Failure to approve might indicate that a hacker is attempting to utilize credentials that have been stolen.

The -audit-policy-file switch may be used to enable audit logging and specify precisely which events should be reported when files are sent to Kube-apiserver. There are four different logging levels you may choose from: None, Metadata only, Request (which logs both requests and responses but not metadata), and RequestResponse (which logs all three). See the documentation for an illustration of an audit policy file.

Managed Kubernetes providers can set up notifications for permission failures and give access to this data via their UI.

8. Maintain Kubernetes Version Consistency

Always use the most recent version of Kubernetes. Here is a collection of known Kubernetes vulnerabilities along with a severity rating.

Always prepare to update to the most recent version of Kubernetes. If you use a hosted Kubernetes provider, see if your supplier supports automated updates. Upgrading Kubernetes might be a challenging procedure.

9. Secure Kubelet

To launch pods and report metrics for nodes and pods, the kubelet—an agent that runs on every node—interacts with the container runtime. You can start and terminate pods and carry out other activities using the APIs that each kubelet in the cluster exposes. The cluster as a whole might be compromised if an unauthorized user obtains access to this API (on any node) and can run code there.

You may use the following configuration parameters to lock the kubelet and lessen the attack surface:

With —anonymous-auth=false, prevent anonymous access so that unauthenticated requests will return an error. The API server must authenticate itself to the kubelet to accomplish this. The flags -kubelet-client certificate and —kubelet-client-key can be added to change this.

• To confirm that requests are approved, change —the authorization mode to a value other than always allow.
• By setting this as a webhook by default, the kubedm installation process makes sure that the kubelet calls Subject Access Review on the API server for authentication.
• To limit kubelet permissions, include NodeRestriction in the API server -admission-control parameter. Due to this, the kubelet can only edit pods that are attached to its node object.
• Close read-only ports by setting —read-only-port=0. As a result, information regarding active workloads cannot be accessed by anonymous users.
• Although it can be exploited during the reconnaissance stage of an attack, this port does not provide hackers access to the cluster.
• Disable cAdvisor, which Kubernetes API statistics have taken the position of as the metrics provider in older versions of Kubernetes. To prevent disclosing details about active workloads, set -cadvisor-port=0.
• For Kubernetes v1.11, this is the default configuration. Use a DaemonSet to launch cAdvisor if you must.

With KSPM (Kubernetes Security Posture Management) and sophisticated agentless Kubernetes Runtime Protection, manage the complexities of Kubernetes security.

To accomplish policy-driven, full-lifecycle protection and compliance for K8s applications, Tools today offers Kubernetes-native capabilities:

• For accurate reporting and correction, Kubernetes Security Posture Management (KSPM) provides a comprehensive picture of the security posture of your Kubernetes environment. assisting you in identifying and addressing security concerns.
• Automate Kubernetes security configuration and compliance - through security assessments and automated compliance monitoring, uncover issues and address them. assist you in implementing policy-driven security governance and monitoring.
• Determine admission of workloads throughout the cluster based on the pod, node, and cluster properties; control pod deployment based on K8s risk. Use pre-built best practices and unique Open Policy Agent (OPA) rules to provide contextual risk reduction.
• Agentless runtime security provides runtime protection for Kubernetes workloads without requiring access to the host operating system, allowing for simple, seamless deployment in managed or restricted K8s settings.
• Open Source Kubernetes Security Tools offers the most well-known open source tools for protecting Kubernetes, such as Kube-Bench, which evaluates Kubernetes clusters against more than 100 tests of the CIS Benchmark, and Kube-Hunter, which conducts penetration tests using numerous well-known attack vectors.

Some Reference Material

• Securing a Cluster | Kubernetes
• Configure a Security Context for a Pod or Container | Kubernetes
• kubectl Cheat Sheet | Kubernetes