All About Rate Limiter

What is Rate Limiter?

A rate limiter is a system component that restricts the rate at which requests are made to a service. There are various approaches to implementing a rate limiter, but one common design is based on the token bucket algorithm.

Here's a high-level overview of the token bucket algorithm:

1. A bucket is initialized with a certain number of tokens.
2. Every time a request is made, a token is removed from the bucket.
3. If there are no tokens in the bucket, the request is denied.
4. Over time, tokens are added to the bucket at a fixed rate.

In a distributed system, multiple instances of the rate limiter can be deployed to handle a large number of requests. The token bucket state can be stored in a shared database like Redis or in a distributed data structure like a distributed cache.

The rate limiter must be thread-safe to ensure that multiple requests can be handled simultaneously. This can be achieved by using locks or atomic operations to update the token bucket state.

In addition to the basic token bucket algorithm, a rate limiter can be designed to handle other features, such as burst protection and different rates for different types of requests.

Overall, the design of a rate limiter depends on the specific requirements of the system, including the scale, latency, and accuracy.

Why do we use Rate Limiter?

Rate limiters are used for several reasons, including:

1. Preventing Denial-of-Service (DoS) Attacks: A rate limiter helps prevent DoS attacks by limiting the rate at which requests are processed by a service. This limits the amount of resources that can be consumed by a single client, making the service more resilient against attacks.
2. Managing Resource Usage: A rate limiter can be used to manage resource usage by limiting the rate at which requests can be made to a service. This helps ensure that the service remains responsive even under high load conditions.
3. Protecting Third-Party Services: When integrating with third-party services, a rate limiter can be used to protect the service from being overwhelmed by a large number of requests. This helps ensure that the third-party service remains available and responsive.
4. Billing: A rate limiter can be used to enforce billing policies by limiting the rate at which requests can be made by a client. For example, a client may be limited to a certain number of requests per month or per second.
5. Fairness: A rate limiter can help ensure fairness by limiting the rate at which requests can be made by a single client. This ensures that all clients have access to the service, even under high load conditions.

Overall, rate limiters are a key component of many services, as they help ensure availability, performance, security, and fairness.

Simple Architecture of a Rate Limiter

As we know the use of rate limiter now let's look at a high-level architecture diagram of a rate limiter system given below:

+-------------------+? ? ? ? +-----------------
|? ?Client Request? |? -->? ?|? ?Load Balancer |
+-------------------+? ? ? ? +-----------------+
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?|
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?v
+-------------------+? ? ? ? +-----------------+
|? ? ?Rate Limiter? |? <--? ?|? ? ?Server? ? ? |
+-------------------+? ? ? ? +-----------------+

+
        

Let's try to know each component of the above diagram:

1. Client Requests: The clients make requests to the service.

1. Load Balancer: The load balancer distributes the incoming requests to one of the servers in the service.
2. Server: The server handles the incoming request and invokes the rate limiter to check if the request is within the allowed rate.
3. Rate Limiter: The rate limiter is responsible for enforcing the rate limits by keeping track of the number of requests made by each client and deciding whether to accept or reject each incoming request.

Above diagram represents a basic architecture of a rate limiter system. In a large-scale system, the rate limiter can be deployed in a distributed manner, with multiple instances running in parallel to handle the incoming requests. The state of the rate limiter can be stored in a shared database like Redis or in a distributed cache.

As now we know what exactly Rate Limiter is Let's try implementing it using node.js. We will be implementing a rate limiter in Node.js using the token bucket algorithm as given below:

Let's create a file RateLimiterApp.js file & paste the below code :

class RateLimiter 
? constructor(rate, burst) {
? ? this.rate = rate;
? ? this.burst = burst;
? ? this.tokens = burst;
? ? this.lastRefill = Date.now();
? }


? tryConsume() {
? ? this.refill();


? ? if (this.tokens > 0) {
? ? ? this.tokens--;
? ? ? return true;
? ? } else {
? ? ? return false;
? ? }
? }


? refill() {
? ? const now = Date.now();
? ? const delta = (now - this.lastRefill) / 1000;
? ? this.tokens = Math.min(this.burst, this.tokens + delta * this.rate);
? ? this.lastRefill = now;
? }
}        

This implementation above uses a class with a constructor that takes the rate and burst values as arguments. The tryConsume method is used to determine whether a request should be allowed. The refill method updates the number of tokens in the bucket based on the time elapsed since the last refill.

The rate limiter can be used in an Express.js application as a middleware to restrict the rate of incoming requests:

const express = require("express")
const app = express();


const rateLimiter = new RateLimiter(10, 100);


app.use((req, res, next) => {
? if (rateLimiter.tryConsume()) {
? ? next();
? } else {
? ? res.status(429).send("Too many requests");
? }
});

;        

In above example, the rate limiter allows up to 10 requests per second, with a burst limit of 100 requests. If the rate limiter returns false from the tryConsume method, the middleware sends a response with the HTTP status code 429 (Too Many Requests).

So finally we have implemented our simple Rate Limiter Application now it's your turn to implement rate limiter for you use case.

At last if you like this article the like, share & subscribe this article. For more contents like those do follow me here Praveen Kumar Byee!! for now.