Demystifying AWS EKS Autoscaling end to end

In this article we will walk through the EKS Autoscaling part on a broader aspects but in a concise manner .

We will cover the following areas in EKS Auto scaling services for broader understanding

1. What is EKS Node Auto-scaling ?

2. What is EKS Pod Auto-scaling ?

3. How EKS node & pod autoscaling works step by step under the hood ?

4. How HPA works ?

5. What is Karpenter ?

6. How Karpenter works in EKS ?

Understanding EKS Auto-scaling: A Comprehensive Guide

1. What is EKS Node Auto-scaling?

EKS Node Auto-scaling, also known as Cluster Autoscaler, dynamically adjusts the number of nodes in your Amazon EKS (Elastic Kubernetes Service) cluster based on the resource requirements of your workloads. It ensures that your applications have the necessary resources to run efficiently without manual intervention.

2. What is EKS Pod Auto-scaling?

EKS Pod Auto-scaling, implemented through the Horizontal Pod Autoscaler (HPA), adjusts the number of pod replicas in a deployment based on observed metrics such as CPU utilization or custom metrics. It ensures that your application can handle varying loads by automatically scaling the number of pods up or down.

3. How EKS Node & Pod Autoscaling Works Step by Step Under the hood ?

EKS Node Autoscaling (Cluster Autoscaler)

Monitoring: The Cluster Autoscaler continuously monitors the cluster's resource usage and pending pods.

Decision Making: When it detects that there are un-schedulable pods due to insufficient resources, it decides to add more nodes.

Scaling Up: It interacts with the underlying infrastructure (e.g., EC2 Auto Scaling groups) to provision new nodes.

Node Registration: New nodes are registered with the EKS cluster and become available for scheduling.

Resource Allocation: The previously un-schedulable pods are scheduled on the new nodes.

Scaling Down: When the resource demand decreases, the Cluster Autoscaler identifies underutilized nodes and scales them down to optimize costs.

EKS Pod Autoscaling (Horizontal Pod Autoscaler - HPA)

Metrics Collection: The HPA continuously monitors the metrics of the pods in a deployment.

Decision Making: Based on the defined threshold (e.g., CPU usage), it decides whether to scale the number of pod replicas up or down.

Scaling Up: If the metrics exceed the threshold, the HPA increases the number of pod replicas to distribute the load.

Scaling Down: If the metrics fall below the threshold, the HPA decreases the number of pod replicas to optimize resource usage.

4. How HPA Works

The Horizontal Pod Autoscaler (HPA) in Kubernetes is a key component for pod autoscaling:

Configuration: Define the HPA resource in a YAML file, specifying the target deployment, the metric to monitor (e.g., CPU), and the desired utilization threshold.

Metrics Server: The HPA relies on the Kubernetes Metrics Server to collect resource usage data.

Reconciliation Loop: The HPA controller runs a reconciliation loop, querying the Metrics Server at regular intervals.

Scaling Decision: Based on the metrics and the defined threshold, the HPA controller decides whether to scale the number of replicas up or down.

Applying Changes: The HPA controller updates the deployment's replica count, which the Kubernetes scheduler enforces.

What is Karpenter? How Karpenter Works in EKS?

Karpenter is an open-source, flexible, high-performance Kubernetes cluster Autoscaler designed to manage dynamic workloads by automatically launching the right compute resources for your EKS cluster.

How Karpenter Works in EKS ?

Event-Driven Scaling: Karpenter reacts to unschedulable pods in real-time, making scaling decisions based on the current state of the cluster.

Optimal Instance Selection: It chooses the most appropriate instance types and sizes based on the resource requirements of the unschedulable pods.

Provisioning Nodes: Karpenter interacts directly with the AWS APIs to provision new instances, optimizing for cost and performance.

Node Registration: Newly provisioned nodes are registered with the EKS cluster and available for scheduling.

Resource Efficiency: Karpenter continuously monitors the cluster and scales down underutilized nodes to ensure cost efficiency.

Conclusion

Autoscaling in Amazon EKS, through both node and pod autoscaling, provides a robust solution for dynamically managing resource requirements, ensuring high availability, and optimizing costs. Tools like HPA and Karpenter play a crucial role in this process, enabling seamless scaling based on real-time metrics and workload demands.