DevOps: "Configuration of "httpd" Docker Container on Target Node running over AWS EC2 Instance using Ansible Playbook"

Ansible is an open-source software provisioning, configuration management, and application-deployment tool enabling infrastructure as code. It runs on many Unix-like systems and can configure both Unix-like systems as well as Microsoft Windows. Wikipedia

Ansible, A powerful IT automation tool as well as the revolutionary invention of the Industry. It can solve almost every task and can reduce time complexity by running a single task on thousands of machines simultaneously.

Docker, another revolutionary invention of the Industry, by integrating it with Ansible, great things can be done.

There is an industry level problem, "configuration of an httpd docker container remotely on target host". There is one of the powerful industry use cases, the given operation can be solved or achieved by following it.

Configuration of Controller Node

In Ansible, There are different types of Nodes - Control (Controller) Node, Managed Node sometimes known as Target Node or Host.

Control Node: Any machine with Ansible installed. Ansible commands and playbooks can be executed by invoking the ansible or ansible-playbook command from any control node. Any computer that has a Python installation can be used as a control node - laptops, shared desktops, and servers can all run Ansible. However, Windows machines can not be used as a control node. We can have multiple control nodes.

Managed Nodes: Manages nodes are the network devices (and/or servers) that can be managed with Ansible. Managed nodes are also sometimes called “hosts”.

To configure the controller node in a machine, we need to follow the below steps

Ansible Installation on EC2 Instance

For Ansible Installation, make sure that the Operating System has Python version 3 installed with pip.

To install Ansible, we need to follow the below steps

Step 1) First of all, we need to run the following pip command to install the Ansible

pip3 install python3

and you may see the following output

WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting ansible
  Downloading https://files.pythonhosted.org/packages/9c/f4/c156b10d7ae90ba6b99b1b126f7d30628adc1e733a6fbd63569852948f21/ansible-2.10.3.tar.gz (28.0MB)
    100% |████████████████████████████████| 28.0MB 47kB/s
Collecting ansible-base<2.11,>=2.10.3 (from ansible)
  Downloading https://files.pythonhosted.org/packages/2e/d1/92422f8f53ae2d4e75ebdc2be2186a9ca2796b5d9679f20ed0239e86d8cf/ansible-base-2.10.3.tar.gz (5.8MB)
    100% |████████████████████████████████| 5.8MB 223kB/s
Requirement already satisfied: jinja2 in /usr/lib/python3.6/site-packages (from ansible-base<2.11,>=2.10.3->ansible)
Requirement already satisfied: PyYAML in /usr/lib64/python3.6/site-packages (from ansible-base<2.11,>=2.10.3->ansible)
Requirement already satisfied: cryptography in /usr/lib64/python3.6/site-packages (from ansible-base<2.11,>=2.10.3->ansible)
Collecting packaging (from ansible-base<2.11,>=2.10.3->ansible)
  Downloading https://files.pythonhosted.org/packages/46/19/c5ab91b1b05cfe63cccd5cfc971db9214c6dd6ced54e33c30d5af1d2bc43/packaging-20.4-py2.py3-none-any.whl
Requirement already satisfied: MarkupSafe>=0.23 in /usr/lib64/python3.6/site-packages (from jinja2->ansible-base<2.11,>=2.10.3->ansible)
Requirement already satisfied: idna>=2.1 in /usr/lib/python3.6/site-packages (from cryptography->ansible-base<2.11,>=2.10.3->ansible)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/lib/python3.6/site-packages (from cryptography->ansible-base<2.11,>=2.10.3->ansible)
Requirement already satisfied: six>=1.4.1 in /usr/lib/python3.6/site-packages (from cryptography->ansible-base<2.11,>=2.10.3->ansible)
Requirement already satisfied: cffi!=1.11.3,>=1.7 in /usr/lib64/python3.6/site-packages (from cryptography->ansible-base<2.11,>=2.10.3->ansible)
Collecting pyparsing>=2.0.2 (from packaging->ansible-base<2.11,>=2.10.3->ansible)
  Downloading https://files.pythonhosted.org/packages/8a/bb/488841f56197b13700afd5658fc279a2025a39e22449b7cf29864669b15d/pyparsing-2.4.7-py2.py3-none-any.whl (67kB)
    100% |████████████████████████████████| 71kB 10.1MB/s
Requirement already satisfied: pycparser in /usr/lib/python3.6/site-packages (from cffi!=1.11.3,>=1.7->cryptography->ansible-base<2.11,>=2.10.3->ansible)
Installing collected packages: pyparsing, packaging, ansible-base, ansible
  Running setup.py install for ansible-base ... done
  Running setup.py install for ansible ... done

Successfully installed ansible-2.10.3 ansible-base-2.10.3 packaging-20.4 
pyparsing-2.4.7

You'll see the Successfully installed ansible in the last.

Step 2) In this additional step, we can verify one more time whether it is successfully installed or not by using the following command as follows

ansible --version

& The following output can be seen

ansible 2.10.3
  config file = None
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.6/site-packages/ansible
  executable location = /usr/local/bin/ansible

  python version = 3.6.8 (default, Dec  5 2019, 15:45:45) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]

Here, it can be seen that "config_file" is set to None. This will be covered in the next section.

Creating Inventory & Ansible Configuration File

An inventory is a list of managed nodes. An inventory file is also sometimes called a “hostfile”. Inventory can specify information like IP address for each managed node. An inventory can also organize managed nodes, creating and nesting groups for easier scaling.

In this section, there are two files to be created - ip.txt & ansible.cfg (inventory file can be created with any name instead of ip.txt)

• ip.txt (Inventory File)
• ansible.cfg (Ansible Configuration File)

Now, to perform this operation, we need to follow the below steps

Creating Inventory File

The default location for inventory is a file called /etc/ansible/hosts. We can specify a different inventory file also.

Step 1) First of all, we need to create a .txt file with any name and anywhere in the Linux Root Directory (Root Directory Recommended) by using the following command

vi /root/ip.txt

& in this file, the following content needs to be written as follows

3.129.210.139 ansible_user=ec2-user ansible_ssh_private_key_file=/root/SSHKeys/shobhitKeyNov2020.pem ansible_connection=ssh

3.14.132.228 ansible_user=ec2-user ansible_ssh_private_key_file=/root/SSHKeys/shobhitKeyNov2020.pem ansible_connection=ssh

Now, there are lots of import configuration can be seen in this file, Let's understand them

First of all, there are two IP addresses that can be seen in this file - 3.129.210.139 & 3.14.132.228. These two IP addresses are the target nodes' public IP addresses and the target nodes are running over EC2 Instances and can be accessed from anywhere in this world. (I've already allowed this in Security Group Policy of EC2 Instance)

There are other 3 fields or variables that can be seen in this file

1. ansible_user - This defines the target node username. In AWS EC2 Instance, the by default username for each EC2 Instances (except Windows) is "ec2-user".
2. ansible_ssh_private_key_file - This defines the private key file which is used for authentication when needed (Because EC2 Instance doesn't support direct passwords, we need to use Private Key File instead of Password to connect Target Node (EC2 Instance) for Ansible. I've used "shobhitKeyNov2020.pem" which is located at the "/root/SSHKeys" custom directory (the directory can be changed according to the user)
3. ansible_connection = This defines the connection type between Managed nodes and the control nodes. I've used "ssh" as the Connection type.

Finally, In this file, I've used my two EC2 Instances as Managed nodes.

Creating Ansible Configuration File

In Ansible Configuration File, we need to specify the ansible hosts or nodes list to be connected with the Ansible control node.

To create Ansible Configuration File, first, we need to create an ansible folder under the /etc/ directory as follows

mkdir /etc/ansible

It will create the ansible folder inside the /etc/ directory & inside the "/etc/ansible" directory, the "ansible.cfg" needs to be created as follows

vi /etc/ansible/ansible.cfg

& inside this file we need to tell the ansible that where the ip.txt (Inventory File) is located as follows

[defaults]
inventory = /root/ip.txt

& after saving it, now we can run the "ansible --version" command to recheck whether the config file is successfully configured or not.

ansible 2.10.3
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.6/site-packages/ansible
  executable location = /usr/local/bin/ansible

  python version = 3.6.8 (default, Dec  5 2019, 15:45:45) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]

Now, it is visible or configured successfully.

Now, we can test it by using the "ansible all -m ping" command whether the managed nodes are accessible and successfully connected or not

ansible all -m ping

The output will be in green color

QUICK FIX

At the time of ping, you might see the following error

3.133.152.84 | UNREACHABLE! => {
    "changed": false,
    "msg": "Failed to connect to the host via ssh: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\r\n@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @\r\n@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\r\nPermissions 0644 for '/root/SSHKeys/shobhitKeyNov2020.ppk' are too open.\r\nIt is required that your private key files are NOT accessible by others.\r\nThis private key will be ignored.\r\nLoad key \"/root/SSHKeys/shobhitKeyNov2020.ppk\": bad permissions\r\[email protected]: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).",
    "unreachable": true

}

To fix this, just run the below command

chmod 600 /root/SSHKeys/shobhitKeyNov2020.ppk

It will fix the issue

Creating Playbook

Playbooks are ordered lists of tasks, saved so we can run those tasks in that order repeatedly. Playbooks can include variables as well as tasks. Playbooks are written in YAML and are easy to read, write, share, and understand.

In this step, We'll create a playbook for the particular operation "Configuration of "httpd" Docker Container on Target Node" and copy the example HTML file in docker httpd from Control Node to Target Node and then Docker Container.

Before the next steps, we need to first create a Static Example HTML file for testing, in my case, I am writing the following syntax

<h1>Hello 0812</h1>

& saved it with "test.html" filename.

To Create Playbook, we need to follow the below steps

Step 1) First of all, we need to create a .yml file (in my case, I am creating it with docker.yml name inside the Playbook directory)

vi /Playbook/docker.yml

Step 2) After that, we need to write the following YAML code for the playbook according to the problem statement.

- hosts: all
  tasks:
  - package:
      name: "docker"
      state: present
    become: true

  - file:
      path: /staticFiles
      state: directory
    become: true

  - copy:
      src: "/root/test.html"
      dest: "/staticFiles/"  
    become: true

  - command: docker pull httpd:latest
    become: true

  - command: docker run -d -p 21800:80 --name HTTP0812Server1 httpd:latest
    become: true

  - command: docker cp /staticFiles/test.html HTTP0812Server1:/usr/local/apache2/htdocs/shobhit.html
    become: true

& save it.

In this Playbook file, there are different modules I've used according to the operation

1. hosts - in this module, I've used "all" which means, when the playbook will execute, this will perform tasks in all the target nodes.
2. tasks - it defines the different tasks according to the requirement.
3. package - this module is used to install packages or software in target node.
4. become - this module is used to execute commands in sudo mode.
5. file - this module is used to create a directory on the target node.
6. copy - this module is used to copy files from the control node to the managed node.
7. command - this module is used to execute user-defined commands in the managed node.

& more.

Finally, the playbook is ready to be used.

Running Ansible Playbook for Automation

So Finally, we can execute our playbook by using the following command

ansible-playbook Playbook/docker.yml

& the output will be

According to this video, the playbook is successfully executed and at the control node the following output can be seen as follows

PLAY [all] *********************************************************************

TASK [Gathering Facts] *********************************************************
ok: [3.129.210.139]
ok: [3.14.132.228]

TASK [package] *****************************************************************
changed: [3.129.210.139]
changed: [3.14.132.228]

TASK [file] ********************************************************************
ok: [3.14.132.228]
ok: [3.129.210.139]

TASK [copy] ********************************************************************
ok: [3.129.210.139]
ok: [3.14.132.228]

TASK [command] *****************************************************************
changed: [3.14.132.228]
changed: [3.129.210.139]

TASK [command] *****************************************************************
changed: [3.129.210.139]
changed: [3.14.132.228]

TASK [command] *****************************************************************
changed: [3.129.210.139]
changed: [3.14.132.228]

PLAY RECAP *********************************************************************
3.129.210.139              : ok=7    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
3.14.132.228               : ok=7    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

That's all

Finally, the conclusion is that using this scenario we can solve this industry-level challenge easily. Now, in the output video, you can see how in a single hit, both the target nodes are configured very quickly, without interacting with them physically and the operations are done simultaneously.

Thanks for reading...

***

This article is written, edited, and published by Shobhit Sharma

Connect with me on Twitter | LinkedIn | Instagram | Facebook | WhatsApp