MlOps Automation Task 2

Problem statement:

1. Create a container image that’s has Jenkins installed using Dockerfile.

2. When we launch this image, it should automatically start the Jenkins service in the container.

3. Create a job chain of job1, job2, job3 and job4 using the build pipeline plugin in Jenkins.

4. Job1: Pull the Github repo automatically when some developers push the repo to Github.

5. Job2: By looking at the code or program file, Jenkins should automatically start the respective language interpreter install image container to deploy code ( eg. If code is of PHP, then Jenkins should start the container that has PHP already installed ).

6. Job3: Test your app if it is working or not.

7. Job4: if the app is not working, then send an email to the developer with error messages.

8. Create One extra job job5 for monitor: If container where the app is running. fails due to any reason then this job should automatically start the container again.

Solution :

I have used the following technologies:

Git: To Obtain the code that the developer commits and if needed in future anyone can modify the developer code. Github link for the Html and PHP codes needed for the project: 

Docker: To create our own customized docker file.

Jenkins: To automate each and everything.

Python script: To send the mail to the concerned person if there's some problem.

RedHat operating system: The operating system in which we create our Dockerfile.

Our first step, creating a Dockerfile, First, we have to create a folder inside the root using the command mkdir task2/. Then go inside this folder using the cd command and create a docker file using gedit Dockerfile command.

In this Dockerfile, I've written multiple commands for various purposes.

I've used here base_os as centos to build our own image using dockerfile.

Now, our Dockerfile is ready. Now we will build our own custom image. Its called build.

Now, we will launch this image so that it will automatically start our Jenkins service in the container. We use the command :

docker run -it --privileged -p 8089:8080 -v /;/base_os --name startjenkins hw:v1

Here, we can also use option -d for launching it in background. So, that we can run further commands. -it is for the interactive terminal, --privileged is for giving permission to access outside world from docker, -p is for patting if we want to run our Jenkins from outside world i.e. from our windows. It means we've to ask to come to the IP of our base operating system.

We will access the Jenkins server from outside. It is called tunnelling. We will use ngrok here. We download the ngrok and unzip it.

We can now use the command ./ngrok http 192.168._._:8089 and use our jenkins from chrome that is installed in our Windows system. 8089 is the port number on which I am running my jenkins. And before it is the IP of our redhat system.

Now, heading towards the jobs : We've to now build the 4 jobs and then we will create a job chain of job1, job2, job3 and job4 using build pipeline plugin.

For our first job, we need gitbash through which we add some files to our github account:

First our job1: It will pull the github repository automatically when some developer pushes the repository to Github. Means it will fetch data from github. So, when developer commits any changes this job should trigger. So I've used here Poll SCM which will monitor github every minute. Our job1 will look like:

Now, our next job is job2. By looking at the code or program file, Jenkins should automatically start the respective language interpreter install image container to deploy code (eg.If code is of PHP, then Jenkins should start the container that has PHP already installed ). Here I am using two types of interpreters - HTML and PHP. For this, we require two different images HTML and PHP. I have created both of them using two different Dockerfiles which are in two different folders.

Now I have build both the html and php images:

So, now our job2:

In the execute shell, I've written the code for 3 types : 1. When only HTML folder exists-

2. If any one of HTML or PHP exists:

3. If both the servers exist then we will write this code in the execute shell:

#if both servers found

if ls | grep html && ls | grep php

then

if docker ps | grep html_product && docker ps | grep php_product

then

echo "Already running both of them"

elif docker ps -a | grep html_product && docker ps -a | grep php_product

then

sudo docker start php_product

sudo docker start html_product

echo "php_product and html_product started"

else

sudo docker run -dti --name html_product -v /fetched_folder:/var/www/html -p 7082:80 html:v1

sudo docker run -dti --name php_product -v /fetched_folder:/var/www/html -p 7083:80 php:v1

echo "php_product and html_product started"

fi

Now, our job3. This job will test our app, if it is working or not.

Now, our last job in the chain. This is job4. If our app is not working, then we've to send email to developer with error msg. For this I've written a code using python script.

If our app has failed, then developer will receive a mail like this:

Now, our pipeline is ready.

After this comes our last job, job5. This job will monitor our container. If container where app is running. fails due to any reason then this job should automatically start the container again. It will check every minute whether container image is present if not it will run the container again.

This was all about my project on automation. This type of project will notify the developer and help him know the error early.

Thank you