Boosting Kubernetes Policy Compliance: Harnessing Kyverno Webhooks for Enhanced Validation
Authored by Anjana Radhakrishnan
Our customer, an e-commerce company, was facing challenges in ensuring consistent resource allocation and efficient resource usage across their Kubernetes cluster. They had a complex microservices architecture with multiple teams deploying and managing applications. Occasional application failures, resource conflict, and performance issues were occurring due to inconsistent definitions of resource requests and limits for containers. Manually monitoring and enforcing resource requirements for each deployment became time-consuming and error-prone.
CloudifyOps recommended a solution by implementing Kyverno, a policy engine, to enforce resource allocation best practices. We created a Kyverno policy called "Require pod requests limits," which ensured that every container had defined values for resource requests and limits. This policy verified that each container specified the required memory and CPU to run and set an upper limit on the resources it could consume. Using Kyverno, we delivered value to our customer by implementing best practices in governance.
Kyverno, an open-source policy engine for Kubernetes, has been designed to enforce and validate Kubernetes configuration. We can deploy the Kyverno policies as a Kubernetes resource. The policy is validated against the Kubernetes deployment, pod, namespace, etc.
How does Kyverno work?
Kyverno works by applying policies to the Kubernetes resources through the admission control mechanism. When a resource is created or modified, Kyverno's webhook endpoints are invoked to intercept and process the requests. Kyverno evaluates policies based on resource information like metadata, labels, annotations, and namespace. Policies are defined with rules containing conditions to be evaluated against the Kubernetes resource. Each rule can have a validate, mutate, generate, or verifyImages declaration.
Types of Kyverno Policy
Policies in Kyverno can be either cluster-wide (ClusterPolicy) or namespaced (Policy). Namespaced policies only apply to resources within the same namespace, and cluster-wide policies are applied to matching resources across all namespaces. Other than this, there is no difference between the two types.
Installing the Kyverno Helm Chart in your Kubernetes Cluster
# Add the Kyverno Helm repository to your repository list helm
repo add kyverno https://kyverno.github.io/kyverno/?
# Scan your Helm repositories to fetch the latest available charts
helm repo update?
# Install the Kyverno Helm chart into a new namespace called "kyverno"
helm install kyverno kyverno/kyverno -n kyverno --create-namespace
A few of the commonly used policies and sample use cases:
Example Policies:
Here is a sample policy sample.yaml, which will restrict the kubernetes resource to be created, when we give the image tag `latest`.
apiVersion: kyverno.io/v1
kind: Policy
metadata:
? name: disallow-latest-tag
? namespace: demo
? annotations:
? ? policies.kyverno.io/title: Disallow Latest Tag
? ? policies.kyverno.io/category: EKS Best Practices
? ? policies.kyverno.io/severity: high
? ? policies.kyverno.io/subject: Pod
? ? policies.kyverno.io/description: >-
? ? ? The ':latest' tag is mutable and can lead to unexpected errors if the
? ? ? image changes. A best practice is to use an immutable tag that maps to
? ? ? a specific version of an application Pod. This policy validates that the image
? ? ? specifies a tag and that it is not called `latest`.? ? ?
spec:
? validationFailureAction: audit
? background: true
? rules:
? - name: require-image-tag
? ? match:
? ? ? resources:
? ? ? ? kinds:
? ? ? ? - Pod
? ? validate:
? ? ? message: "An image tag is required."
? ? ? pattern:
? ? ? ? spec:
? ? ? ? ? containers:
? ? ? ? ? - image: "gcr.io/istio-testing/proxyv2 | *:*"
? - name: validate-image-tag
? ? match:
? ? ? resources:
? ? ? ? kinds:
? ? ? ? - Pod
? ? validate:
? ? ? message: "Using a mutable image tag e.g. 'latest' is not allowed."
? ? ? pattern:
? ? ? ? spec:
? ? ? ? ? containers:
? ? ? ? ? - image: gcr.io/istio-testing/proxyv2:latest |? !*:latest"
Apply the policy in the Kubernetes cluster.
Kubectl apply -f sample.yaml
After applying the Kyverno policy, create a deployment in the kubernetes cluster which has the `latest` image tag.
kubectl create deployment nginx --image=nginx:latest
Here, we can see the Kyverno policy is restricting the Kubernetes deployment from getting created, as we have mentioned the rules in the policy.
Now we can create the deployment again with a known registry.
kubectl create deployment nginx --image=nginx:1.25.0
Benefits achieved:
Improved Resource Efficiency - 20% decrease in resource consumption: The customer noticed a big improvement in using resources more efficiently when they enforced resource requests and limits. This resulted in better use of available resources and prevented wasting resources.
Enhanced Application Stability - 30% decrease in application failures and 15% reduction in incidents related to performance: The application's stability improved because resources were consistently allocated according to set limits. This had a positive effect on the customer's experience and minimized the disruptions caused by application downtime.
Cost Savings - estimated 15% cost reduction: The customer achieved cost savings by using Kyverno's policy enforcement to optimize resource allocation and avoiding unnecessary overprovisioning.
Time and Effort Savings - 40% reduction: Automating the enforcement of resource allocation policies with Kyverno significantly reduced the manual effort required for monitoring and enforcing resource requirements. The customer’s DevOps teams could focus on strategic initiatives and improving development velocity.
Compliance and Governance: Kyverno offered the customer a centralized and consistent method to enforce resource allocation policies throughout their Kubernetes cluster. This assisted them in meeting compliance requirements and maintaining a governance framework, ensuring they followed the best practices for resource allocation.
Overall, Kyverno policies provide a powerful mechanism for managing and enforcing policies within Kubernetes, promoting cluster stability, security, and compliance with organizational standards.
To know more about how the CloudifyOps team can help you optimize your resource utilization, write to us today at [email protected].
Reference: