Securing Kubernetes using kube-bench
Center of Internet Security(CIS) Benchmark provides an objective, consensus-driven security guideline for the Kubernetes Server Software. kube-bech from aquasecurity is a tool that checks whether Kubernetes is deployed securely by running the checks documented in the?CIS Kubernetes Benchmark.
kube-bench checks whether Kubernetes is deployed according to security best practices as defined in the CIS Kubernetes Benchmark. kube-bench keeps up the pace with CIS Benchmarks as well as kubernetes versions.
Benchmark components
kube-bench offers to scan the configurations of various components kubernetes listed below.
1.Master node
- apiserver
- scheduler
- controllermanager
- etcd
- flanneld
- kubernetes(config file /etc/kubernetes/config)
- kubelet
2. Worker Node
- kubernetes(config file /etc/kubernetes/config)
- kubelet
- proxy
3. etcd
4. controlplane
5. policies - Cluster level configurations to validate
6. managedservices - Platform specific configuration checks in aks, eks etc.
Note: Managed kubernetes such as AKS,EKS etc would not allow to bench mark the control plane/master node. However it is possible to benchmark the worker nodes only.
kube-bench constantly updates the policies pertaining to a managed kubernetes and you will be able to review the configuration options on config file of kube-bench on github.
Installing kube-bench
kube-bech can be run as pod probably as kubernetes job, OR binary executable directly at host level. It is also possible to install kube-bench using container to a linux node.
Install kube-bench from a container
Run the below command to install kube-bench on the node.
docker run --rm -v `pwd`:/host aquasec/kube-bench:latest install
Install kube-bench from installer
$ wget https://github.com/aquasecurity/kube-bench/releases/download/v0.6.5/kube-bench_0.6.5_linux_amd64.rpm
$ rpm -ivh kube-bench_0.6.5_linux_amd64.rpm?
rpm: RPM should not be used directly install RPM packages, use Alien instead!
rpm: However assuming you know what you are doing...
Preparing...? ? ? ? ? ? ? ? ? ? ? ? ? ################################# [100%]
Updating / installing...
? ?1:kube-bench-0:0.6.5-1? ? ? ? ? ? ?################################# [100%]
$ kube-bench version
0.6.5
$
On Ubuntu node, you may install dpkg as well.
领英推荐
naveen@DESKTOP-SS1RTGI:~$ wget https://github.com/aquasecurity/kube-bench/releases/download/v0.6.5/kube-bench_0.6.5_linux_amd64.de
naveen@DESKTOP-SS1RTGI:~$sudo dpkg -i kube-bench_0.6.5_linux_amd64.de
Selecting previously unselected package kube-bench.
(Reading database ... 82382 files and directories currently installed.)
Preparing to unpack kube-bench_0.6.5_linux_amd64.deb ...
Unpacking kube-bench (0.6.5) ...
Setting up kube-bench (0.6.5) ...
naveen@DESKTOP-SS1RTGI:~$ kube-bench version
0.6.5
naveen@DESKTOP-SS1RTGI:~$
How to run benchmark
kube-bech can be run as pod probably as kubernetes job, OR binary executable directly at host level. For both approaches, the utility would need access config files and running process.
Run benchmark from node
Running kube-bench from a node would require kubeconfig as well as the configuration of kube-bench(yaml) which describes the tests which are to be executed.
./kube-bench --config-dir `pwd`/cfg --config `pwd`/cfg/config.yaml
Run as Pod
Running kube-bench as pod would require host level filesystem access to check configuration files as well as host level PID namespace to check the running processes. For starters, you may try running the Kubernetes job from kube-bench github.
PS C:\Users\navee> kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job.yaml
job.batch/kube-bench created
PS C:\Users\navee> kubectl.exe get pod
NAME? ? ? ? ? ? ? ?READY? ?STATUS? ? ? RESTARTS? ?AGE
kube-bench-gw7jv? ?0/1? ? ?Completed? ?0? ? ? ? ? 112m
PS C:\Users\navee> kubectl.exe logs kube-bench-gw7jv
[INFO] 1 Master Node Security Configuration
[INFO] 1.1 Master Node Configuration Files
[PASS] 1.1.1 Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Automated)
[PASS] 1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Automated)
[PASS] 1.1.3 Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Automated)
.....
[Redacted]
Reviewing the benchmark result
Benchmark utility would provide the security controls scanned and provides remediations if there are any divergence in desired configuration.
PS C:\Users\navee> kubectl.exe logs kube-bench-gw7jv
[INFO] 1 Master Node Security Configuration
[INFO] 1.1 Master Node Configuration Files
[PASS] 1.1.1 Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Automated)
[PASS] 1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Automated)
[PASS] 1.1.3 Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Automated)
.....
[Redacted]
.....
== Remediations master ==
1.1.12 On the etcd server node, get the etcd data directory, passed as an argument --data-dir,
from the below command:
ps -ef | grep etcd
Run the below command (based on the etcd data directory found above).
For example, chown etcd:etcd /var/lib/etcd
1.1.21 Run the below command (based on the file location on your system) on the master node.
For example,
chmod -R 600 /etc/kubernetes/pki/*.key
.....
[Redacted]
.....
== Summary master ==
45 checks PASS
10 checks FAIL
10 checks WARN
0 checks INFO
[INFO] 2 Etcd Node Configuration
[INFO] 2 Etcd Node Configuration Files
[PASS] 2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Automated)
[PASS] 2.2 Ensure that the --client-cert-auth argument is set to true (Automated)
[PASS] 2.3 Ensure that the --auto-tls argument is not set to true (Automated)
[PASS] 2.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Automated)
[PASS] 2.5 Ensure that the --peer-client-cert-auth argument is set to true (Automated)
[PASS] 2.6 Ensure that the --peer-auto-tls argument is not set to true (Automated)
[PASS] 2.7 Ensure that a unique Certificate Authority is used for etcd (Manual)
== Summary etcd ==
7 checks PASS
0 checks FAIL
0 checks WARN
0 checks INFO
.....
[Redacted]
.....
Note: Managed kubernetes such as AKS,EKS etc would not allow to bench mark the control plane/master node. However it is possible to benchmark the worker nodes.
Conclusion
kube-bench helps to test the consensus-driven security guideline for the Kubernetes Server Software offered by CIS benchmarking. In a CI pipeline, JUnit based output and scored based on CIS standard helps to measure the configuration of a cluster and improvise it.
Reference:
Up-next
In forthcoming articles, I will be covering on how to run kube-bench on managed kubernetes cluster(i.e. AKS) as well as running the benchmark in CI pipeline(Azure DevOps).
Cyber Security Engineer || DevSecOps Architect || AI DLP Researcher || Application Security || Cloud Security || API Security ll Kubernetes Security || VAPT || DevSecOps || CCNA-R&S,CCNP-SECURITY,CEHv11,AZ-900,CKA,AZ-901
2 年Nice one.. Have you written the next part yet ?
Head of Engineering @ LV= | Azure, AWS - DataOps/DevOps/MLOps, Cross-functional Team Leadership
3 年Few questions I have: 1. Is Kube bench run in specific namespace and can you pick and choose it’s scope by adding labels to namespace ? 2. Does it spins off a sidecar container with your application pods ? 3. Dashboards are via Prometheus and grafana for the kube bench testing or it has its own centralised console which can be accessed using security token login just like Kubernetes dashboard?