How to Load Balance Kibana

As the user base grows and the use cases expand, the need to scale Kibana becomes increasingly important.

High Availability

• Single Point of Failure: Without load balancing, if a single Kibana instance fails, the entire visualization and analysis capability of your Elastic Stack can be disrupted.
• Continuous Access: Load balancing ensures that if one instance goes down, others can still serve the traffic, maintaining uninterrupted access to Kibana.

Scalability

• Traffic Distribution: As user demand grows, a single Kibana instance may become a bottleneck. Load balancing distributes traffic across multiple instances, allowing Kibana to handle more simultaneous users and larger query volumes.
• Horizontal Scaling: Easily add more Kibana instances to the load balancer to scale out the infrastructure, enhancing the overall capacity without overhauling the existing setup.
• Alerting: Kibana alerts are crucial for monitoring and responding to changes in your data. As your use of Kibana expands, the volume and complexity of alerts can increase, necessitating scaling strategies to handle these effectively.

For deployement, we have used three c6g.large EC2 instances running Debian 12 on AWS.

wget https://artifacts.elastic.co/downloads/kibana/kibana-8.14.1-amd64.deb
sudo dpkg -i kibana-8.14.1-amd64.deb        

and bunch of other commands following the official documentation here, we had kibana running and connected to elasticsearch.

For Alerting, kibana requires additional step, i.e. adding encryption key as documented here. The encryption key needs to be added to kibana config, for debian, configuration file is located at:

/etc/kibana/kibana.yml

At the end of above file, add a random key of 32 characters or more example generator.

xpack.encryptedSavedObjects.encryptionKey: <A string of 32 character or more>        

Now we have alerting setup done.

Steps to be done before creating a image for load balancing:

1. Add security key to config as we did for encryptedSavedObjects above

xpack.security.encryptionKey: something_at_least_32_characters        

This ensures, when a user's request is load balanced, to different kibana instance, the user is not logged out. The details can be found here.

2. Shutdown kibana service

sudo systemctl stop kibana.service        

3. Enable kibana service to be started on boot

sudo systemctl enable kibana.service        

4. Enable kibana to be accessible to load balancer by changing the key in kibana.yml

server.host: "0.0.0.0"        

5. Delete kibana uuid file should be in one of the below locations

sudo rm /var/lib/kibana/uuid        

or

sudo rm /usr/share/kibana/data/uuid        

6. Now proceed with creating an image, and attaching it to load balancing target group.

Summary

Deleting the uuid file and adding xpack.security.encryptionKey are the only additional steps required to load balance kibana.