Ever think about how load balancer picks the node to invoke.?

Load balancer plays a crucial role in identifying the node which fulfills client request.

Ex: We visited a garage where few technicians are available to work on our car. In this case Garage Manager will assign a technician who can take care of our car.        

There are several standard algorithms which helps in deciding the node by the load balancer.

Round robin: It was simple First Come First Serve.

Example: Manager will assign requests in a sequential order. At the end of technicians, will start with first technician again.        

Sticky Round robin: Extension of round robin. Send subsequent request from the same user will send to same server.

Example: Manager will assign requests in a sequential order. But the same technician will be assigned to the customer who visited second time onwards.        

Weighted Round robin: Requests will be served based on weightage which was configurable.

Example: Manager will assign a rating to every technician. Highest rated technician will be given first priority.        

Hash based: Performs some mathematical computation called hashing and then deriving a number, which is then mapped to individual server. Based on this number request will be redirected

Hashing can be performed by several factors like IP, URL etc.

Azure uses five-tuple hash algorithm.

Example: Manager defines his own logic based on some factors like experience, accuracy and service and will come to a conclusion to which technician should work on the repair.         

Least connection: Dynamic algorithm, in which the request will be redirected to server which is having least number of active connections.

Example: Manager identifies which technician has less number of in completed repair cases, then add the current one to the respective technician.        

Least Response Time: Dynamic algorithm, in which the request will be redirected to server which is having faster response time.

• It considers the historical performance of servers to make decisions about where to route incoming requests, optimizing for faster processing.
• The dynamic aspect comes from the continuous monitoring of server response times and the adaptive nature of the algorithm to route incoming requests to the server with the historically lowest response time

Example: Manager identifies which technician who takes less amount of time to complete a repair, then add the current one to the respective technician.        

Please note that every algorithm above has pros and cons. There is no limitation that one can use any of this, we can write our own implementation as well. :)

Please comment which one you prefer most for your application.

#loadbalancer #systemdesign #algorithm