DEVOPS ASSEMBLY LINE PART-2

Hello guys hope you enjoy my first article part-1 in this we're going to create a pipeline which give us faster/agility deployment of website in production server. now i'm here with some another best practice/approach in DevOps.

TECHNOLOGY USED:-

• GIT
• GITHUB
• JENKINS
• RHEL 8 (RED HAT ENTERPRISE LINUX VERSION 8)
• DOCKER

So in this article i come with new problem which can be solved by DevOps approach, so let me explain you what we are going to do, as far we know Docker is so fast even we can create, deploy, and run applications very easily. Docker basically a product which work on Containers technology and it allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and deploy it as one package, so by this we can create our environment very fast.

Now first we are going to create a full fledged image which contain jenkins software and other dependencies software with the help of DOCKER FILE. so when you use the image it will setup the jenkins environment for you.

These are the instructions what we are going to perform :-

1. We've to create an container image that has jenkins installed by using docker files & after build when we launch this image, it should automatically start jenkins service in the container.
2. Then i will create four jenkins job, here are the brief about four jobs :-

• JOB1- This job will pull the GitHub repository when developer commits.
• JOB2- After pull the repository this JOB2 will check the file/code/ if the code is in php format then this job2 start the php container, so this JOB2 automatically launch the respective language interpreter image and deploy the code/file in it.
• JOB3- This job is for checking, after deploying the code might be chances code is not working properly! so if server/application is not responding/working then this job will send email to developer team with error message as well as success message if it is succeed.
• JOB4- This job to monitor the container if any container goes down due to any reason this job will re-launch the container again.

Now what is Docker File? Dockerfile is a script, composed of various commands (instructions) and arguments, let me explain briefly if you want to create a custom own docker image there are two ways to create a Docker image:

1:- Manually using the docker commit command

2:- Automatically using a Dockerfile.

So here we are using Dockerfile, Docker file use Go programming language, so it has their own syntax i will explain you below..

#vim Dockerfile 

( Here i used vim text editor it is available in linux or you can use gedit,vi,nano. here 'Dockerfile' is predefined filename. )

FROM <image name> 

( It defines that image we going use to start the build process. It can be any image, including the ones you have created previously. If a FROM image is not found on the host, Docker will try to find it (and download) from the Docker Hub or other container repository. It needs to be the first command declared inside a Dockerfile. )

RUN <command> 

( This syntax is used to run the command during developing the image, this can help us by executing/running command to form the image. )

CMD <command> 

( This syntax is sometimes similar as RUN it is also used for executing the command/argruments difference between RUN and CMD that CMD is not executed during build, an example for CMD would be running an application upon creation of a container which is already installed using RUN [e.g. yum install python36 -y] then we can run CMD python36. )

COPY             

( Adds files from your host to image directory The COPY instruction copies new files or directories from <src> and adds them to the filesystem of the container at the path <dest>. ) 

For more info about Dockerfile you can visit here

Now from here our practical start...................................

#mkdir task2 / (create a workspace)

#cd task2 / ( change directory to task2 )

after creating task2 folder i created another folder name 'dockerrepo' inside task2 folder & copied the docker repository file from my host system to this folder because i want to install docker inside my custom image..

command: cp /etc/yum.repos.d/docker.repo  /root/task2/dockerepo

#vim Dockerfile 

(here i used vim text editor it is available in linux or you can use gedit,vi,nano. here 'Dockerfile' is predefined filename )

after saving the file i run the docker command for building the image

#docker build -t <any image name>:<any version name> <directory path>

in my case i run #docker build -t jenkins-docker:v1 . (here dot means current directory)

now it will takes time to build our custom image depend on your internet speed.

Okay its time to run our custom image.

#docker images 

( By this command of docker you can see your all images present in your system also you can see our custom build images )

As my requirement i want to launch docker inside docker ! so here i used a approach where inside container named 'jenkins-docker' docker daemon doesn't independent means inside container docker doesn't have their own daemon, it is connected to the main host RHEL docker daemon or we can say connected to the daemon of docker in host system. and we having docker command line interface inside container as well as on host both are connected to each other & Docker clients inside the containers will be using the Docker daemon from the host.

after successfully building and launching the container you will see this type of screen here "beecc" type string is your initial admin password yes later you can change it. i explain how we can change our default password of jenkins in previous blog :)

#ifconfig  

(check your ip of your system then type on your browser with jenkins expose port in my case 1337 )

Just copy and paste your initial admin password and continue. in the next page you will see there is option "install suggested plugins" you can select this option it will download all suggested plugins from internet make sure you have internet connectivity.

If you want to manually install plugins you can do by going to manage jenkins>manage plugins>available section > search your desired plugins

now i am going to create repository on github and also local git repository for know more how to create visit to my previous DevOps Assembly line part-1 :)

so from here i am going to jump on job1

• JOB1- This job will pull the GitHub repo when developer commits..

in Source Code Management section select Git.. In repository URL give your GitHub remote repository url...

in branch specifier i give "dev1"  

in execute shell i copy all the files from jenkins workspace to folder /root/task2/job1

?In Build Triggers section i select first Trigger builds remotely and give Authentication Token "redhat" this command (curl --user "admin:admin" https://192.168.43.43:8080/job/job3/build?token=redhat) so this command i give to the developer team where he can put this code inside in his local git repository configuration folder .git/ > hooks> post-commit... as soon as developer commit this code will trigger to job1 to run automatically.......

job1 completed........ now its time to job2

• JOB2- After pull this JOB2 will check the file/code/ if the code is in php format then this job2 start the php container, if the code in python,html then this JOB2 automatically launch the respective language interpreter image and deploy the code/file in it...

In Build Triggers section i select 'Build after other projects are built' this options help us for chaining so i select 'job1' if previous job/job1 successfully run then this job2 will run.

in the execute shell i give some if and else statement + some linux and docker commands let me explain all these 

so our requirement is if the code is in php format then this job2 start the php container, if the code in python,html then this JOB2 automatically launch the respective language interpreter image and deploy the code/file in it..

for this here i used if statement for finding the .php extension file inside the workspace of jenkins in my custom container ( grep is a Linux / Unix command-line tool used to search for a string of characters in a specified file.)

if this command find .php extension file then echo "php code found" (here echo is for printing the string)
again i used if statement with some docker command to find the name of container called php_container if php_container exist then start this container.. 

('docker ps -a' is the docker command for finding/listing the process status of container -a means all )  

else statement (here if 'php_container'  is not running/available then run a container i assign name called 'php_container' here i used pre-installed php intepreter and httpd server image for launching our php environment website, here -p option is used for exposing the container to outside world )

at last i used  bash script, using using docker cp for each multiple file if .php  multiple extension file found then copied to php_container server document root directory... here i used (docker cp command this command is used for Copy files/folders between a container and the local filesystem)

fi ( end of statement )

as i explained above for php i used same approach for python and html file too... for learning docker commands visit https://docs.docker.com/

now our job 2 is completed :) now job 3

• JOB3- This job is for checking , after deploying the code it is working or not! if server/application is not responding/working then this job will send email to developer team with error message as well as success message if it is succeed

now this job is for checking and notify the client via email so for emailing the admin we have a some setup/configuration in jenkins, so let me explain you by picture how you can do it....

First go to homepage/dashboard of jenkins click on manage jenkins.

then go to 'Configure System'

Scroll down below to page you will find E-mail Notification give your credentials and configuration settings which i mentioned in picture below.

Apply>Save Now your Email-Notification configuration is Ready :)

Now create your JOB3

Chaining complete for job3 if previous job2 successfully run then this job2 will run.

In the Execute shell i have done some testing of my containers that is my server is running properly or not! let me explain you how i tested!

first i go to inside containers where jenkins is running...

#docker exec -it <container-name> /bin/bash 

#docker inspect < container name > ( this command give us all the information about container like ip address, image used, id etc Docker inspect provides detailed information of container  and it result/output in JSON format )

so i give --format flag (it is used for the output using the given Go template simply we use for printing the strings )

after this i used JSON parser for retrieving the information what i need ) and my requirement is to take only ipaddress of the container 

"{{.NetworkSettings.IPAddress}}" (here we use curly bracket because  'NetworkSettings' is the main variable and inside this we have 'IPAddress' variable,  it is type of multi-dimensional array) and i used '.' dot before 'NetworkSettings' is part of nothing/empty so do is a parser.

so overall this whole command give me only ip address of container then i saved the output in 'docker_ip_php' variable 

after this i checked by curl command and giving the variable 'docker_ip_php' instead of ip address/url because as i mention above 'docker_ip_php' have ip address of php_container..

so how i test it is working or not?  .. i used HTTP headers concept  i use  command curl -i <ip>:<port> (The curl command transfers data to or from a network server, ( It is designed to work without user interaction, so it is ideal for use in a shell script ) so i only concern about HTTP header by this i am checked my server is working or not....as you can see in picture above it give me 200 means  HTTP 200 OK success status response code indicates that the request has succeeded.

exit 0 ( Every Linux or Unix command executed by the shell script or user has an exit status You normally use exit 0 if everything went ok.)

exit 1 ( But if your program detects an error and decides to abort, you would use       exit 1 )

By this way i am testing my server and deployment of website is working properly or not! question? why i used this approach instead of variable i can use directly ip of container right ! . answer is whenever container goes down due to any reason and again restarted might be chances ip will be change but container name will be same... so instead of directly putting the ip in curl i assign a variable for it ....

After execute-shell i go to Post-build Actions section and put my email address... SAVE>APPLY :) JOB3 completed.........................................

Now its time to our last job job4

• JOB4- This job to monitor the container if any container goes down due to any reason this job will re-launch the container again...

in job4 you can see i used to watch job3 successfully built then job4 will run also i used POLL SCM ! what is poll scm ? i explain in my previous blog you can check ! well in this i give H/5 * * * * it means this job automatically run every 5 minutes...so if any container goes down this job will launch it again...

As you can see above it is similar type of code written in job2 but here i am looking for running container if sudo docker ps | grep php_container is running then print "php server running" else create a container name called 'php_container' also copy the respective multiple source files to server document root ...this approach i used for python and html too...

now our job4 is completed...................................... :)

`NOW ITS TIME TO CREATE A BUILD PIPELINE WHERE YOU CAN VISUALISE YOU JOB AND ALSO IT IS INTERACTIVE

Go to jenkins homepage/dashboard>

Go to Plus sign .

go to 'View name' give any name you can give.. select 'Build Pipeline View' click OK

In Upstream/downstream config section select initial job as job1> Apply> OK

now you can see a interactive visual of your pipeline.............. as soon as developer commit his code it will automatically run :)

Build Pipeline :) from here you can run your job and you will see a pipeline of job

also i get the successfull message via jenkins in my mail !

Now by browser you can open the link and see the website..

PHP one

HTML one

............................................................The End..........................................................................

So now you have successfully build our automation pipeline as well as your personal image of docker which contain jenkins and lots of dependencies software , then we integrate with various tools like docker, git, github, rhel os .... if you have any doubt, queries you can ask in comment box :)

Thank you for reading :) i hope you all like my article..

?