eBPF Stateful Programs and State Synchronization Problem

Like any other stateful application, eBPF programs store their state in eBPF Maps. However, to avoid a single point of failure, these applications are often deployed across multiple nodes, in a so called high-availability setup.

But how do we ensure that decisions like IP blacklisting or DNS client rate-limiting remain consistent across all nodes when each node maintains its own eBPF Map to track client request counts?

In today’s newsletter, we’ll explore the problem of state synchronization of eBPF Maps across multiple nodes.

The Problem

Today, eBPF programs are widely used for stateful networking solutions such as:

• Load Balancing: Storing information about backend servers, including IP addresses, ports, and health states, to choose one for forwarding incoming traffic.
• Connection Tracking: Storing timestamps for connection activity. If a connection remains idle for too long, the eBPF program can clean up old entries, effectively implementing a garbage collection mechanism for expired sessions.
• Firewalls: Storing state information about active connections, such as the number of packets or bytes associated with an IP address within a time window, to dynamically enforce rate limits or blacklist clients.

Unlike stateless applications, high-availability stateful applications often need to maintain consistent state information across all nodes in a cluster. In the scenario of eBPF application, the state of each node's eBPF Map must be synchronized across the cluster.

?? eBPF maps are key-value data structures used to store and share data between eBPF programs and user-space applications or across different eBPF programs.

However, there is currently no known synchronization tool or daemon available for eBPF Maps.

The Solution

To address this, I decided to build a solution myself.

Here’s a high-level overview of the solution:

This approach leverages asynchronous eBPF map notification updates...

Read the full post, on my Substack Newsletter: https://ebpfchirp.substack.com/p/ebpf-stateful-programs-and-state