MLops-task2: Complete Automation using Jenkins, Docker, Git&GitHub Integration

Jenkins:

Jenkins is an open source automation server. With Jenkins, organizations can accelerate the software development process by automating it. Jenkins manages and controls software delivery processes throughout the entire lifecycle, including build, document, test, package, stage, deployment, static code analysis and much more.

Docker:

Docker is a tool designed to make it easier to create, deploy, and run applications by using containers. Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and deploy it as one package.

GIT&GitHub:

Git is a version control system that lets you manage and keep track of your source code history. GitHub is a cloud-based hosting service that lets you manage Git repositories. If you have open-source projects that use Git, then GitHub is designed to help you better manage them. 

Task description:

1. Create container image that’s has Jenkins installed using dockerfile 

2. When we launch this image, it should automatically starts Jenkins service in the container.

3. Create a job chain of job1, job2, job3 and job4 using build pipeline plugin in Jenkins 

4. Job1 : Pull the Github repo automatically when some developers push repo to Github.

5. Job2 : By looking at the code or program file, Jenkins should automatically start the respective language interpreter install image container to deploy code ( eg. If code is of PHP, then Jenkins should start the container that has PHP already installed ).

6. Job3 : Test your app if it is working or not.

7. Job4 : if app is not working , then send email to developer with error messages.

8. Create One extra job job5 for monitor : If container where app is running. fails due to any reson then this job should automatically start the container again.

Before doing the task, we require few things:

a] Redhat or any linux OS (RHEL 8 in my case) b] Account on GitHub

STEP 1:

First we need to create a workspace, and in that workspace we have to create a dockerfile. This dockerfile after build will be a docker image, and what this image will do is that after container is launched, Jenkins will automatically start running.

To build the docker image, following is the command:

command: docker build -t <image_name:version> [PATH]

STEP 2:

After building the image, we have to launch a container using docker run command. As we also will need to launch container inside this Jenkins container therefore we have two ways to achieve such a configuration. Either we can use Docker Sockets or we can use “Execute Shell Script on Remote host using SSH”. For SSH, we need to install the SSH plugin and then set the remote host and other information.

I am here going to use the Docker Sockets. For this, I am going to mount the sockets of my base OS (RHEL 8) with the container that is going to be launched. 

We can also use -d option if we want our container to run in background.to check if our container has launched or not by,

command: docker ps -a

STEP 3: Jenkins setup

When you login first time to the Jenkins. Jenkins will provide you one password you can use that password for the login to the Jenkins dashboard. After doing login you can set your own password. Here i am not going to set the password. I will use the password that is provided by Jenkins.

The command for going inside and getting bash shell along with the path for password is given below:

You can change your password after completing the setup in settings. (manage Jenkins>manage users>change password)

All the required plugins are installed during setup, but if something is missed then we can install it from settings.

STEP 4: Jobs for our task

JOB 1:

In this task, we have to pull the code from GitHub repository. For this we require, Git plugins which I have installed. Here I have done Poll SCM, meaning Jenkins will keep on checking repo if some new code has been uploaded or not. We can use git hooks also.

A new directory is created where all the code will be copied.

Console output for JOB 1:

JOB 2:

In this job Jenkins will automatically launch a container having respective language interpreter. In my case since the code is html, so the container which will be launch should have html interpreter, for that I have used httpd image.

Also after launching the container, I am copying my code from Jenkins container to the location form where the new container will read the webpage/code.

This job will be triggered by JOB 1.

Console output for JOB 2:

JOB 3 & JOB 4:

This job will check if our website is working fine or not by retrieving the satus code of website. If status code is 200 then it means website is there and if other codes like 404 means page not found, 000 means client abort i.e connection refused from client (I received this status code few times during testing).

JOB 3 will be triggered by JOB 2.

If the site is not working, then JOB 4 (which is included in the same job as JOB 3) will send a mail to the developer regarding this. To use the email-notification, we need to install the plugin first and configure it. Go to manage Jenkins>configure system

Console output for JOB 3 & JOB 4:

Output of email received when job is unsuccessful:

JOB 5:

This job will keep on monitoring the container with html interpreter, and if for some reason the container gets terminated, then this job will automatically start the container once again.

JOB 5 will be triggered by JOB 3, and it will keep on running as long as we need the environment to deploy our applications.

Console output for JOB 5:

Build Pipeline view:

FINAL OUTPUT: Successful working of site

GitHub link: