Why do we need to secure Kubernetes?

Here, I have covered major security threats to an application running on Kubernetes Cluster.

Below listed are the 'Threats' followed by 'Mitigation Techniques':

1. A malicious user having shell/bash access, can take the shell of running container. A malicious user can make rogue/unauthorised API calls to Kube API server (from the container) and can extract the cluster specific information - Disable shell/bash access for the running containers. Remove binary executable files from the container bin directory (we can configure Dockerfile file accordingly) to ensure that application cannot be re-executed after making the unauthorised changes in them. Ensure that while deploying service accounts for each deployment (within a specific namespace), the auto-mount service account token is set to false (automountServiceAccountToken: false). This ensures that Kube-API calls are blocked from the container to the K8s cluster.
2. A malicious user can extract the service-account tokens or manually created secrets (running on Kubernetes cluster) from the ETCD database where secrets are not stored in encrypted format (within ETCD Database). A malicious user has to make a simple API call to the kube API server to extract the sensitive details. For eg: “ETCDCTL_API=3 etcdctl get /registry/secrets/default/secret1 [...] | hexdump -C” - Ensure that all the secrets (including service accounts) are stored in encrypted format in the ETCD database. Refer?https://v1-23.docs.kubernetes.io/docs/tasks/administer-cluster/encrypt-data/?to enable encryption of all secrets. Do not forget to recreate all the secrets again, once the encryption is enabled. Otherwise, none of your existing secrets would be recognized by the k8s cluster. Refer below command to recreate secrets: Refer kubectl get secrets --all-namespaces -o json | kubectl replace -f -
3. A malicious user can infiltrate the deployment files, having access to 'Default' namespace and therefore also has access to 'Default' ServiceAccount. A malicious user can delete and re-deploy the deployment files with unauthorised changes having adverse effects - Ensure that deployment files are configured based on their specific service accounts. Service accounts should be mapped to the specified namespace. Service accounts should be bound with RBAC permissions by applying appropriate Role and RoleBinding.
4. A malicious user can extract sensitive information stored in ConfigMap - Ensure that we must have a cloud vault integration (like hashicorp) in place and all the sensitive information should be stored in the cloud vault. To decrease the severity level to medium, we should deploy the sensitive information as a secret rather than deploying it as a ConfigMap on K8s cluster. But, please note that deploying the sensitive information as a secret will only ‘base64 encode’ the sensitive information/secrets on the k8s cluster, which could still be exploited.
5. A malicious user can extract the sensitive information from the common configuration-files (ConfigMap) running in ‘Default’ namespace - Ensure that ConfigMaps, Secrets, Services, Deployments and PersistentVolumeClaims are deployed in a specific namespace rather than running them in the default namespace. This ensures that only specified name-spaced deployments can access the resources. Ensure appropriate labels are assigned to each resource for restricting access to the resources, based on specific labels only. Ensure that we have RBAC policies with appropriate Role and RoleBinding configured for the admins/users/apps accessing specified namespace.
6. A malicious user can perform MITM or can extract the sensitive information if TLS is not enabled for the application - Implement Secure Ingress resource by specifying a 'Secret' that contains a TLS private key and certificate. The Ingress resource only supports a single TLS port, 443, and assumes TLS termination at the ingress point (However, traffic to the Service and its Pods is in plaintext).
7. A malicious user can make system calls / kernel calls from the running containers, having root privileges - Ensure that containers are running with non-root privileges. Ensure that below is set for deployment files, to restrict root calls from running containers: runAsNonRoot: true, privileged: false, allowPrivilegeEscalation: false, readOnlyRootFilesystem: true, capabilities: drop: - all. As we are setting the containers to runAsNonRoot: true, we must even set the security contexts for deployments as non-root. For eg: securityContext: runAsUser: 1001 runAsGroup: 2000 fsGroup: 3000. Use Seccomp profiles- Seccomp stands for secure computing mode and has been a feature of the Linux kernel since version 2.6.12. It can be used to sandbox the privileges of a process, restricting the calls (system calls) it is able to make from userspace into the kernel. Refer:?https://v1-23.docs.kubernetes.io/docs/tutorials/security/seccomp/? Use AppArmor profiles - AppArmor is a Linux kernel security module that supplements the standard Linux user and group based permissions to confine programs to a limited set of resources. AppArmor can be configured for any application to reduce its potential attack surface and provide greater in-depth defense. Refer:?https://v1-23.docs.kubernetes.io/docs/tutorials/security/apparmor/
8. A malicious user can perform DOS attack by sending large no. of API requests to running containers. This would lead the containers to perform ‘noisy neighbours attack’ and can exhaust all the CPUs and Memory of other running pods/containers - Ensure that you must assign resource quota policies at the namespace level, in order to limit overconsumption of the CPU and memory resources a pod is allowed to consume.
9. A malicious user can take advantage of misconfigured security-configurations within the yaml deployment files, deployed on the Kubernetes cluster. Security misconfigurations could lead to vulnerabilities within the system - Ensure that yaml files are scanned with tools like Kubesec (https://github.com/controlplaneio/kubesec) to secure the yaml file configurations. Security misconfigurations report can be extracted by running the command: “docker run -i kubesec/kubesec:v2 scan /dev/stdin < k8s/specified.yaml ”. Based on the findings, fix all the security misconfigurations present in the yaml files.
10. A malicious user can exploit the vulnerabilities found in open sourced docker images - Ensure that docker images are scanned for vulnerable components using an image scanning tool like trivy (https://github.com/aquasecurity/trivy). Vulnerability report can be extracted by running the command: “docker run aquasec/trivy image specifiedimage:version”. Based on the findings, do not use the vulnerable docker images. Use alternative images which do not have any vulnerabilities.
11. A malicious user can perform MITM or can extract the sensitive information if TLS is not enabled for the Kubernetes Dashboard Access - Limit the exposure of Kubernetes Dashboard. Disable public access via the internet. Ensure the Dashboard Service Account is not open and accessible to users. Configure the login page and enable RBAC. Configure TLS for accessing Dashboard.
12. A malicious user can exploit Kubernetes-Cluster security misconfigurations and can perform a variety of attacks based on misconfigured settings. Security misconfigurations lead to vulnerabilities within the system - Ensure that all the Kubernetes nodes (Master node & worker nodes) are scanned separately with the Kube-bench tool to identify security misconfigurations. Make sure to fix all the security misconfigurations based on the findings/report. Refer:?https://github.com/aquasecurity/kube-bench?Harden the cluster nodes to avoid security attacks (Refer https://cloud.redhat.com/blog/12-kubernetes-configuration-best-practices#9-secure-the-configuration-files-on-the-master-node ), secure the Worker node configuration files (Refer https://cloud.redhat.com/blog/12-kubernetes-configuration-best-practices#12-secure-the-worker-node-configuration-files) and also harden other components of K8s cluster.
13. A malicious user could perform an unauthorised action and can deny its occurrence after completion of that unauthorised action - In the absence of proper auditing in place for all the actions on k8s cluster, it might not be possible to prove the occurrence of an event, and also trace the events leading to an untoward/malicious event. Ensure that we implement Falco as a service on the K8s cluster and configure necessary alerts for auditing of activities on K8s cluster. Some examples of events that should trigger an alert would include: A shell is run inside a container, A container mounts a sensitive path from the host such as /proc, A sensitive file is unexpectedly read in a running container such as /etc/shadow, An outbound network connection is established. Refer Falco Project:?https://falco.org/docs/
14. A malicious user can pull and execute rogue docker images which are not listed in the organisation's private registry. A malicious user can execute the latest or beta tagged images and can invite zero day attacks on k8s cluster - Ensure that we have mechanisms in place to restrict developers to only run whitelisted images from organisation’s private repository. We can implement Open Policy Agent (OPA) to restrict developers not to use: Images with latest/beta/insecure tag etc Pull images from company’s local registry only OPA is highly scalable. We can also configure OPA to enforce other restrictions on k8s cluster. Refer:?https://github.com/open-policy-agent/opa
15. A malicious user can create and execute the deployment files with root privileges. A malicious user can mount root file system/directories into the containers - Implement Pod Security Policies (PSP) on K8s cluster, to restrict admins/developers from running the applications with root privileges. We can implement PSP to resolve below security aspects: Do not run application processes as root, Do not allow privilege escalation, Use a read-only root filesystem, Do not use the host network or process space, Drop unused and unnecessary Linux capabilities, Use SELinux options for more fine-grained process controls, Give each application its own Kubernetes Service Account, Do not mount the service account credentials in a container if it does not need to access the Kubernetes API. Refer:?https://v1-23.docs.kubernetes.io/docs/concepts/security/pod-security-policy/?https://cloud.redhat.com/blog/12-kubernetes-configuration-best-practices#2-use-pod-security-policies-to-prevent-risky-containerspods-from-being-used
16. A malicious user can shell into the application container & can perform root permissive/ escalated tasks to get hold of the host kernel (bare metal kernel access) on which the K8s cluster is running - Ensure that applications are running as sandboxed pods to increase the level of security within K8s cluster. Sandboxed pods use a different runtime class environment (refer?https://v1-23.docs.kubernetes.io/docs/concepts/containers/runtime-class/) rather than the one getting used by K8s cluster. This separates/sandbox the kernel level access for pods/containers running on K8s cluster. Refer?https://gvisor.dev/docs/?https://katacontainers.io/docs/
17. A malicious user can spin up a container & can execute privileged tasks (as root). A malicious user can abuse security misconfigurations within a Docker file & can perform a variety of attacks based on misconfigured settings - Ensure that Dockerfiles do not run with a root user. Ensure that Dockerfiles are scanned with Docker Benchmark script. Ensure that all the security misconfigurations/findings are addressed, found in docker benchmark scan report. Refer:?https://github.com/docker/docker-bench-security
18. A malicious user can exploit the vulnerabilities existing in Kubernetes image files - Update K8s to the latest version whenever possible. Every new release is typically bundled with a host of different security features. Scan K8s images regularly. Verify the hashes before running the K8s images.
19. A malicious user can shell into the Kubernetes cluster containers to exploit cluster specific services - Disable shell access for all the non-required cluster components (bash as well as sh access).
20. A malicious user who has access to corporate network, can exploit or get hold of Kubernetes cluster pods - Kubernetes nodes must be on a separate network and should not be exposed directly to public networks. This is only possible if Kubernetes control and data traffic are isolated. Otherwise, both flow through the same pipe, and open access to the data plane implies open access to the control plane. Ideally, nodes should be configured with an ingress controller, set to only allow connections from the master node on the specified port through the network access control list (ACL).