Automating Load Balancing Infrastructure with Ansible and HAProxy on AWS??

Load Balancing is an essential component of modern web application architecture. It allows you to distribute traffic across multiple servers, improving the overall performance and availability of your application. HAProxy is a popular open-source load balancer that provides high performance, reliability, and scalability. In this article, we will learn how to set up a HAProxy load balancer on AWS using Ansible.

USE CASES OF LOAD-BALANCERS ??

1. Web Application Load Balancing: HAProxy is commonly used to distribute incoming traffic across multiple web servers, which helps to improve application performance, reduce downtime, and provide high availability. This is especially useful for websites that experience high levels of traffic, such as e-commerce sites or social media platforms.
2. Database Load Balancing: HAProxy can be used to distribute database queries across multiple database servers, which helps to improve performance, ensure scalability, and provide redundancy. This is especially useful for applications that require a high level of database access, such as financial or healthcare applications.
3. API Load Balancing: HAProxy can be used to distribute incoming API requests across multiple API servers, which helps to improve API performance, reduce latency, and provide high availability. This is especially useful for companies that rely on APIs for their core business functions, such as payment processing or logistics tracking.
4. Microservices Load Balancing: HAProxy can be used to distribute traffic across multiple microservices, which helps to improve application performance, reduce downtime, and provide high availability. This is especially useful for companies that have adopted a microservices architecture, as it allows them to manage and scale individual services independently.
5. Hybrid Cloud Load Balancing: HAProxy can be used to distribute incoming traffic across multiple cloud providers, which helps to improve application performance, reduce latency, and provide redundancy. This is especially useful for companies that have adopted a hybrid cloud approach, as it allows them to manage and scale their infrastructure across multiple cloud providers.

Need to know more about Ansible check the below ??

Objectives

1. Provision EC2 instances through ansible.
2. Retrieve the IP Address of instances using the dynamic inventory concept.
3. Configure the web servers through the ansible role.
4. Configure the load balancer through the ansible role.
5. The target nodes of the load balancer should auto-update as per the status of the web servers.

Prerequisites

Before we begin, make sure you have the following:

• An AWS account
• Ansible installed on your local machine or on a VM

Grab a cup of coffee ??and get comfortable because we're about to embark on a wild ride.

PART 1: Create IAM Roles to access EC2

• Login to AWS and create di and rootforansible IAM user di and rootforansible

IAM??Users??Add

• also, create an access key and secret key and save it for further use.

PART 2: Configure INVENTORY

• Starting with separate workspace ?? # mkdir HAProxy

• create ansible.cfg

[defaults]
inventory = /root/Projects/HAproxy/inventory
host_key_checking = False
roles_path =? /root/Projects/HAproxy/roles
private_key_file = /root/Projects/newKey.pem
remote_user = ec2-user
ask_pass = false




[privilege_escalation]
become = true
become_user = root
become_method = sudo
become_ask_pass = false

        

• run # ansible --version
• download ec2.py and ec2.ini in inventory directory

 $ wget https://raw.githubusercontent.com/ansible/ansible/stable-2.9/contrib/inventory/ec2.py        

$ wget https://raw.githubusercontent.com/ansible/ansible/stable-2.9/contrib/inventory/ec2.ini        

Make the ec2.py file executable using chmod

$ chmod +x ec2.py        

The python script has variables for aws region, access id and secret key of the AWS account. We can set it using the export command.

$ export AWS_REGION='ap-south-1'
$ export AWS_ACCESS_KEY_ID='< di user's access key>'
$ export AWS_SECRET_ACCESS_KEY='<di user's secret key>'        

run and check output ./ec2.py --list

this gives no IPs because there are no instances launched.

PART 3: Instance Provisions

• Create role ec2provison in the roles directory

$ ansible-galaxy init ec2-provison        

• code for /roles/ec2provision/tasks/main.yml

• Here you can see I am using the ec2 module for launching the ec2 instance, and I am using variables for different keywords, at last, I provided AWS access and secret key for authentication to my AWS account.
• Now open vars/main.yml with a text editor

--
# vars file for ec2provision
aws_instance_type: "t2.micro"
ami_id: "ami-0e742cca61fb65051"
aws_instance_tags: "webserver"
subnet_id: "subnet-0bd997bad3cccf64b"
aws_region: "ap-south-1"
security_group_id: "sg-0c52c10420d2fa934"
        

• Create security group with inbound rule for All traffic
• In /vars/ also create a ansible vault to store access key and secret key of IAM role rootforansible

ansible-vault create credntialskeys.yml        

Set Password and remember

NOW CREATE A PLAYBOOK TO LAUNCH INSTANCES

vim createInfra.yml

- hosts: localhost
? vars_files:
? ? - /root/Projects/HAproxy/roles/ec2provision/vars/credntialskeys.yml
? roles:


? ? - role: ec2provision        

Run Playbook createInfra.yml and give password

ansible-playbook --ask-vault-pass createInfra.yml        

It will give o/p like

You can also check was console

Now list Hosts

ansible all --list-hosts        

PART 4: CREATE ANSIBLE ROLE TO CONFIGURE WEBSERVERS

• create role webserver

ansible-galaxy init webconfig        

In webconfig/tasks/main.yml write below code

# tasks file for webconfig
- name: install httpd server
? package:
? ? name: "httpd"
? ? state: present
? register: httpd_status


- name: copy web pages
? template:
? ? src: "index.html"
? ? dest: "/var/www/html/index.html"


- name: start web service
? service:
? ? name: "httpd"
? ? state: started

        

goto roles/webconfig/templates/

put/ create your index.html file or webpage

PART 5: CREATE ANSIBLE ROLE TO CONFIGURE LOAD-BALANCER

• Create an ansible role to configure haproxy load balancer software
• Create the following tasks in?load_bal role in?tasks/main.yml?file.

# tasks file for load_bal
- name: install haproxy
? package:
? ? ?name: "haproxy"
? ? ?state: present


- name: cp lb config file
? template:
? ? ? src: "haproxy.cfg"
? ? ? dest: "/etc/haproxy/haproxy.cfg"
? notify: restart_lb


- name: start load_balancer
? service:
? ? name: "haproxy"
? ? state: started

        

In this playbook's tasks we installing HAProxy, Configuring and Starting service

• Create ansible handler to restart haproxy service. handlers/main.yml

- name: restart_lb
? service:
? ? ?name: "haproxy"
? ? ?state: restarted        

• Create the?haproxy.cfg file,?write the following for loop code to?auto-update this file dynamically when any new webserver ip is added in web_instances group in inventory.

create haproxy.cfg at /HAproxy/roles/load_bal/templates

#--------------------------------------------------------------------
# Example configuration for a possible web application.? See the
# full configuration options online.
#
#? ?https://www.haproxy.org/download/1.8/doc/configuration.txt
#
#---------------------------------------------------------------------


#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
? ? # to have these messages end up in /var/log/haproxy.log you will
? ? # need to:
? ? #
? ? # 1) configure syslog to accept network log events.? This is done
? ? #? ? by adding the '-r' option to the SYSLOGD_OPTIONS in
? ? #? ? /etc/sysconfig/syslog
? ? #
? ? # 2) configure local2 events to go to the /var/log/haproxy.log
? ? #? ?file. A line like the following can be added to
? ? #? ?/etc/sysconfig/syslog
? ? #
? ? #? ? local2.*? ? ? ? ? ? ? ? ? ? ? ?/var/log/haproxy.log
? ? #
? ? log? ? ? ? ?127.0.0.1 local2


? ? chroot? ? ? /var/lib/haproxy
? ? pidfile? ? ?/var/run/haproxy.pid
? ? maxconn? ? ?4000
? ? user? ? ? ? haproxy
? ? group? ? ? ?haproxy
? ? daemon


? ? # turn on stats unix socket
? ? stats socket /var/lib/haproxy/stats


? ? # utilize system-wide crypto-policies
? ? ssl-default-bind-ciphers PROFILE=SYSTEM
? ? ssl-default-server-ciphers PROFILE=SYSTEM


#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
? ? mode? ? ? ? ? ? ? ? ? ? http
? ? log? ? ? ? ? ? ? ? ? ? ?global
? ? option? ? ? ? ? ? ? ? ? httplog
? ? option? ? ? ? ? ? ? ? ? dontlognull
? ? option http-server-close
? ? option forwardfor? ? ? ?except 127.0.0.0/8
? ? option? ? ? ? ? ? ? ? ? redispatch
? ? retries? ? ? ? ? ? ? ? ?3
? ? timeout http-request? ? 10s
? ? timeout queue? ? ? ? ? ?1m
? ? timeout connect? ? ? ? ?10s
? ? timeout client? ? ? ? ? 1m
? ? timeout server? ? ? ? ? 1m
? ? timeout http-keep-alive 10s
? ? timeout check? ? ? ? ? ?10s
? ? maxconn? ? ? ? ? ? ? ? ?3000


#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend main
? ? bind *:5000
? ? acl url_static? ? ? ?path_beg? ? ? ?-i /static /images /javascript /styleshe? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ets
? ? acl url_static? ? ? ?path_end? ? ? ?-i .jpg .gif .png .css .js


? ? use_backend static? ? ? ? ? if url_static
? ? default_backend? ? ? ? ? ? ?app


#---------------------------------------------------------------------
# static backend for serving up images, stylesheets and such
#---------------------------------------------------------------------
backend static
? ? balance? ? ?roundrobin
? ? server? ? ? static 127.0.0.1:4331 check


#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend app
? ? balance? ? ?roundrobin
{% for hosts in groups['tag_Name_webserver'] %}
? ?server app1 {{ hosts }}:80 check
{% endfor %}        

bind *:5000

this line is binding port 5000 of load_balancer to reverse-proxy to send traffic to backend servers

{% for hosts in groups['tag_Name_webserver'] %}
? ?server app1 {{ hosts }}:80 check
{% endfor %}        

this loop binds our managed hosts to the backend

PART 5: CREATE A MAIN.YML PLAYBOOK

To run ?webconfig and load_bal role

- hosts: tag_Name_webserver
? roles:
? ? ? ? ? - role: webconfig


- hosts: tag_Name_LoadBalancer
? roles:
? ? ? ? ? - role: load_bal        

RUN

ansible-playbook main.yml        

and final output will be

To Check LB is Working or Not

Goto as per above o/p put IP of balancer to browser with binding port

and refresh it will give route you to these 3 backend servers

https://13.235.50.62:5000        

1st backend server

refresh page

2nd backend server

refresh

3rd and last

That's all for now. Keep learning, keep growing, and keep being awesome!