Automated deployment of webservers and Haproxy Load Balancer ..using Ansible!

Statement: Deploy a Load Balancer and multiple Web Servers on AWS instances through ANSIBLE!

Task Description..!

?? Provision EC2 instances through ansible.

?? Retrieve the IP Address of instances using the dynamic inventory concept.

?? Configure the web servers through the ansible role.

?? Configure the load balancer through the ansible role.

?? The target nodes of the load balancer should auto-update as per the status of web servers.

Summary: One-Click Instance Launched, Web Servers provisioned and Load Balancer ready!

Prerequisites..

1.Ansible installed-(controller node): In order to configure anything using ansible it is foremost to have the controller node(The node on which the ansible is installed).

2.Aws Credentials(access and secret keys) : In this task we are going to provision the instances on AWS ,so we need the credentials(Given by IAM service)to login to aws account and continue with the process.

Before starting the task I believe you might know the basic terms of ansible if not then go through this link.

Now let's start the task ,learn and implement.

1.Ec2 Instances provision..

Firstly, inorder to make ansible to contact AWS we need to install boto libraries of python.

To install : *pip3 install boto3*

Playbook for provisioning 3 ec2 instances(later we use 2 for webservers and one for loadbalancer) :

?? Whenever we run any playbook we need some managed node where we can execute and configure but in this scenario we don't have any managed we are provisioning the os's..so in this case we can use our localhost(Controller node) to contact aws.*

ec2p.yml

- hosts: localhost
  vars_files:
          - awskeys.yml
          - var.yml
  tasks:
          - name: Provisioning 3 instances on AWS..!
            ec2:
                   key_name: "{{key_pair}}"
                   instance_type: "{{instance_type}}"
                   image: "{{ami_image}}"
                   wait: yes
                   vpc_subnet_id: "{{subnet}}"
                   assign_public_ip: yes
                   region: "{{region}}"
                   state: present
                   aws_access_key: "{{aws_access_key}}"
                   aws_secret_key: "{{aws_secret_key}}"
                   instance_tags:
                           Name: "{{item}}"
            loop: "{{instances}}"
            register: total

In the above playbook , I used two files namely awskeys.yml and var.yml to keep my aws credentials and variables as follows:

Now, for credentials first we create a file with the keys and enter the values of it ..

And now as these credentials are very critical for us ...we need to secure them from others in opening it. For this in ansible we have the concept of vault ...to secure our files.

Ansible-vault : Ansible Vault encrypts variables and files so you can protect sensitive content such as passwords or keys rather than leaving it visible as plaintext in playbooks or roles. To use Ansible Vault you need one or more passwords to encrypt and decrypt content.

?? With the help ansible-vault command-line tool we can create and view encrypted variables, create encrypted files, encrypt existing files, or edit, re-key, or decrypt files. You can then place encrypted content under source control and share it more safely.

Now, to encrypt the existing file i used the following command,

*ansible-vault encrypt awskeys.yml*

Now,let's run the playbook and launch our instances..

To run: ansible-playbook --ask-vault-pass ec2p.yml

Playbook run successfully and here is the result of our step1..!

2.Retrieving the Ip adresses dynamically.

In this step we need to retrieve all the ip's of our provisioned instances dynamically using the concept of dynamic inventory. Let's see how it is.

?? Need of dynamic inventory: In a configuration – especially a cloud setup such as AWS where the inventory file keeps constantly changing as you add or decommission servers, keeping tabs on the hosts defined in the inventory file becomes a real challenge. It becomes inconvenient going back to the host file and updating the list of hosts with their IP addresses.And this is where a dynamic inventory comes to play.

?? what is a dynamic inventory? : A dynamic inventory is a script written in Python, PHP or any other programming language. It comes in handy in cloud environments such as AWS where IP addresses change once a virtual server is stopped and started again.

That is in order to work on dynamic inventory , ansible has precreated scripts for almost all platforms.. here we apply the same.

?? Download Inventory scripts to inventory directory using wget command:

$wget https://raw.githubusercontent.com/ansible/ansible/stable-2.9/contrib/inventory/ec2.py

$wget https://raw.githubusercontent.com/ansible/ansible/stable-2.9/contrib/inventory/ec2.ini

?? In ec2.py edit the first line as shown:

?? Make the scripts executable..!

$ chmod +x ec2.py 
$ chmod +x ec2.ini

?? Export your credentials by using the export command:

$ export AWS_REGION='ap-south-1'
$ export AWS_ACCESS_KEY_ID='ADB4XXXXXXXXXXXXXXX'
$ export AWS_SECRET_ACCESS_KEY='EF2@RTXXXXXXXXXXXXXXXX'

Now, we are set to retrieve the ip's of our instances from aws..

Run : ansible all --list-hosts

Based on the above ip's and our requirement I made the following changes:

?? Inventory file:

Note: *Here I added the ip's to groups so we can perform tasks respectively...But here I added manually and if you want to make it automated again we have another module named "add_host" and we have to use this module in the playbook which we used for provisioning the instances and also we can run further plays on these hosts in that same playbook only. To implement refer this link.*

?? ansible.cfg file:

Now, we have our nodes ready ...let's start configuring.!

3.Configure webservers using ansible role.3.Configure webservers using ansible role.

?? In General, ansible role is nothing but the way of organizing our playbooks with multiple directories...for example a role consist of multiple directories namely tasks(which consists of a file named main.yml to write all the tasks to be executed),vars directory for variables, similarly for handlers and templates etc..

Ansible role: Roles let you automatically load related vars_files, tasks, handlers, and other Ansible artifacts based on a known file structure. Once you group your content in roles, you can easily reuse them and share them with other users. An Ansible role has a defined directory structure with seven main standard directories.

Now, let's create roles for webserver and loadbalancer...use the commands :

$ ansible-galaxy init loadbalancer
$ ansible-galaxy init webserver

Note: Run the above two commands from the path /etc/ansible/roles/ ...which is the path from where ansible search for roles by default.

Now, lets work on webserver role:

tasks->main.yml:

Similarly, we can customize more using rest directories like vars and etc.. and for basic webserver it's enough as above.

3.Configure Haproxy-loadbalancer using ansible role.

?? What is Load Balancing ?: Load balancing refers to efficiently distributing incoming network traffic across a group of backend servers, also known as a server farm or server pool.

?? What is Load Balancer ?: A load balancer acts as the “traffic cop” sitting in front of your servers and routing client requests across all servers capable of fulfilling those requests in a manner that maximizes speed and capacity utilization and ensures that no one server is overworked, which could degrade performance. If a single server goes down, the load balancer redirects traffic to the remaining online servers. When a new server is added to the server group, the load balancer automatically starts to send requests to it.

?? What is HAProxy Load Balancer ?: HAProxy is free, open source software that provides a high availability load balancer and proxy server for TCP and HTTP-based applications that spreads requests across multiple servers.

Now, let's write in the loadbalancer role:

tasks->main.yml

handlers->main.yml

templates : In this directory I have placed the haproxy configuration file with some changes made into it so that it can automatically detect the new webservers launched and start distributing load it. I get this file by installing haproxy for testing and now it will be uploaded to the node which we configure as loadbalancer using template module.

Changes made in haproxy.cfg file:

5.Final Playbook to configure both webservers and loadbalancer..using roles.

Now, we have to write a playbook to implement the both roles..!

infra.yml :

Now, as a final step we just need to run this playbook to configure our complete infrastructure...

??? And that's done let's look at our website...to access the website we use the public ip of the loadbalancer on port number 8080.

Here is the result...!

From the above result, we can observe that the load has been distributed between two servers ...

And Finally That's all for the task..! This task has been completed under the mentorship of Mr.Vimal Daga.

Thanks for reading..! Keep connected to me LinkedIn for more and incase of suggestions...Thank you.?? ?? ??