Kubernetes-based Event Driven AutoScaling - KEDA

Introduction

Scaling in Kubernetes is reactive, based on the CPU and memory consumption of a container. However, sometimes the scaling needs to be event-driven. Event-driven architectures are the next evolution of microservices, providing a flexible and decoupled design. Microsoft and Red Hat partnered to build this open source component to provide event-driven capabilities for any Kubernetes workload.

KEDA - more

KEDA allows for fine-grained autoscaling (including to/from zero) for event-driven Kubernetes workloads. KEDA serves as a Kubernetes Metrics Server and allows users to define autoscaling rules using a dedicated Kubernetes custom resource definition.

KEDA can run on both the cloud and the edge, integrates natively with Kubernetes components such as the Horizontal Pod Autoscaler, and has no external dependencies.KEDA enables any container to scale from zero to potentially thousands of instances based on event metrics like the length of a Kafka stream or an Azure Queue. It also enables containers to consume events directly from the event source instead of decoupling with HTTP. KEDA can drive the scale of any container and is extensible to add new event sources.

KEDA performs two key roles within Kubernetes. First, it acts as an agent to activate and deactivate a deployment to scale to and from zero on no events. Second, it acts as a Kubernetes metrics server to expose rich event data like queue length or stream lag to the horizontal pod auto-scaler to drive scale out. It is up to the deployment to then consume the events directly from the source. This preserves rich event integration and enables gestures like completing or abandoning queue messages to work out of the box.

Today KEDA and supporting tooling works with Kafka, Azure Queues, Azure Service Bus, RabbitMQ, HTTP, and Azure Event Grid / Cloud Events. More triggers will continue to be added in the future including Azure Event Hubs, Storage, Cosmos DB, and Durable Functions. KEDA event sources are extensible

Summary

KEDA provides a FaaS-like model of event-aware scaling, where deployments can dynamically scale to and from zero based on demand. KEDA also brings more event sources to Kubernetes. While other Kubernetes offerings have attempted to bring events from other sources, they usually involve converting the event to an HTTP request, which means data and context is lost in the process and direct communication with the event source is limited. KEDA unlocks rich connectivity directly to the event sources, enabling rapid and proactive scaling while preserving direct interaction with the event. Lastly, KEDA is open source on GitHub and open to community contributions.