Test and Merge new developer's code in Master Branch for Production using Automation with Devops Practices

Hello Folks, in the era of competitive world we need to be smart and swift to survive. In IT industry there are so many issues we face in day to day life which will make us less efficient. So in this article, I came up with the solution for one of the most faced issue in software industry.

Problem Scenario :

There is a developer who code in his system using Git for Version controlling. He has launched his application. But there is a need to modify this application by same Developer or any new developer. So the new code which this will be written cannot be deployed without testing. So using Git, GitHub, Jenkins, Docker we are going to fulfill this requirement.

• There is a Master branch in Git on Developer's system in which production ready (tested) code is present. If some new code is added either by any new developer or main developer then we can not add it to production environment without testing.
• So for this we need to create new git branch named as NewDeveloper. When new developer add some code in this branch then, first it need to be tested then merged to master and will be deployed in Production Environment.

Solution :

To implement this scenario, I have created a solution by integrating GitHub, Jenkins and Docker. This diagram is the high level architecture for the solution. This can be implemented by creating 4 Jobs in Jenkins. Each job is described below. I have differentiated these jobs with different colors in diagram.

Start - This flow starts when developer in NewDeveloper branch add some code and commit. Commit will automatically push the code in GitHub NewDeveloper Branch and will fire Jenkins Job 1.

• Job 1 (Red) - This job will go to NewDeveloper branch in GitHub and Fetch the code in Jenkins workspace. Then it will create new Testing Environment using Container and deploy the code in it. When this job builds successfully then Job 2 will fire.
• Job 2 (Green) - This job will perform Automation Testing on the code. If code fails then automatically mail will be sent to the New Developer, or if code passed then it will fire Job 3.
• Job 3 (Brown) - This job will merge the NewDeveloper branch with Master branch for deployment of code in production. When this Job builds successfully then Job 4 will start.
• Job 4 (Blue) - When the NewDeveloper branch is merged to Master branch then it will fetch complete code from Master Branch and create Production Environment (if not exist) and Deploy the code by which client can access the application.

Implementation

Prerequisite :

# Git Bash must be there is the Developer System with a working Repository on GitHub too.

For Git Setup you can refer from this link

https://help.github.com/en/enterprise/2.17/user/github/importing-your-projects-to-github/adding-an-existing-project-to-github-using-the-command-line

# For testing and production environment Linux (Red Hat) machine with Docker & Jenkins installed is needed.

Jenkins - https://www.jenkins.io/download/

Docker - https://docs.docker.com/get-docker/

...... (1) IMPLEMENTING JOB 1 ......

Steps : -

1.1) Create a new Branch in Git using Git Bash named as NewDeveloper.

1.2) Checkout to New Developer then edit code and save.

1.3) Create new Job Named as newDeveloper and Click on configure Job.

# Here GitHub Plugin must be installed in Jenkins

1.4) Go on SCM and add your GitHub repo URL and Enter name of newDeveloper branch.

1.5) Now Click on Build Trigger and Copy the URL with entering access code. This url will be added in git hooks file named as "post-commit" by which this Job will be triggered remotely.

The script in post-commit file will run when user commit the code then first it will push the code on GitHub and then Job 1 will Triggered Remotely. We have automated these steps.

#!/bin/bash
echo "new commit is done"
git push
curl --user "admin:redhat" https://192.168.43.18:8080/job/newDeveloper/build?token=newDev

1.6) Go to Build > Add Build Step > Select Execute Shell > Enter complete script as given below to deploy the code in Testing Environment > Save

1.7) As all these processes will run on the Jenkins server so we need to make Jenkins as a superuser by making its entry in /etc/sudoers file.

 jenkins      ALL=(ALL)    NOPASSSWD: ALL

...... (2) IMPLEMENTING JOB 2 ......

Steps : -

2.1 Create new Job (Job 2) named as newDevTest and configure the Job. Go on Build Triggers >> Check "Build after the other projects are build" >> Enter newDeveloper (Job1) name >> Trigger only if build is stable.

2.2 Now Go on Build Section >> Execute Shell >> Write Conditional Test script like this.

if
curl https://192.168.43.18:8084
then
sudo python3 /mailthis.py
else
curl --user "admin:redhat" https://192.168.43.18:8080/view/Devops%20Solution/job/MergeBranches/build?token=merge
fi

This script will first test the code and if code fails then mail will be sent to Developer by running mailthis.py python file. This script must be saved in the specified location (/mailthis.py) on Jenkins Server.

import smtplib, ssl


port = 587  # For starttls
smtp_server = "smtp.gmail.com"
sender_email = "[email protected]"
receiver_email = "[email protected]"
password = 'yourpassword'
message = """\
Subject: Final Model Build 
Pull it from GitHub along with history file, that is saved using pickle module
"""
context = ssl.create_default_context()
with smtplib.SMTP(smtp_server, port) as server:
    server.ehlo()  # Can be omitted
    server.starttls(context=context)
    server.ehlo()  # Can be omitted
    server.login(sender_email, password)

    server.sendmail(sender_email, receiver_email, message)

And if code pass the test then next Job will be fired using remote trigger URL of next Job by which code will be merged.

......(3) IMPLEMENTING JOB 3 ......

Steps : -

3.1) Create new Job named as MergeBranches >> Configure >> Source Code Management >> Enter URL of GitHub Repo and Credentials (ID password) of GitHub Account >> Specify new Developer branch (which is to be merged).

3.2) Now go on Additional Behaviours >> Add >> Merge Before Build >> Add Name of Repository (origin) >> Branch Name to be merged with (master).

3.3) Go on Build Triggers (As in previous job we have to trigger this merging right after the code run successfully in "2.2") >> Select Trigger Builds Remotely >> Enter Authentication token >> Paste it into script to trigger builds remotely.

3.4) Go on Posts-build Actions >> Add post-build action >> Select Git Publisher >> Check Merge Results >> Enter path of branch to push.

This will merge the newDeveloper Branch with Master Branch by which the complete tested code is now in Master Branch.

......(4) IMPLEMENTING JOB 4 ......

4.1) Here we need to fetch the code from GitHub Master Branch and deploy that code on web server Environment. Create New Job (WebServer) >> Source Code Management >> Enter Your Repository URL >> Specify master branch name.

4.2) In Build Trigger Section - > Check "Build after other projects are Built" >> Enter Job 3 that is "MergeBranches".

Check Poll SCM -> Schedule in a way that in every minute Jenkins will check on GitHub for new Code and if new code encountered the job will be build.

Here we can also use GitHub hook trigger for GITScm polling.

4.3) Here go on build section >> select Execute Shell and write script for creating Web server (if not present) and deploy code.

sudo cp -v * /webServerCode
if sudo docker ps | grep finalWebServer
then
echo "already running"
elif sudo docker ps -a | grep finalWebServer
then
sudo docker rm -f finalWebServer 
else
sudo docker run -d -t -i -p 8085:80 -v /webServerCode/:/usr/local/apache2/htdocs/ --name finalWebServer httpd
fi

So all the configurations and steps for creating the solution is over.

Now we're ready to run the complete setup to use our solution.

......(5) How to run ......

Steps : -

5.1) Okay! not steps its just a step. The new Developer has to write and commit his code.

Here after commiting every thing will be Automated.

This is the output which client can see before newDeveloper commit.

What will happen after Commit : -

• After the commit, code will be automatically pushed to the GitHub newDeveloper branch and Job 1 will be fired (Jenkins Job 1 - newDeveloper).

• Job 1 will deploy the code in Testing Environment. This is the output which is to be tested.

• Here in this setup i have only written one test case, otherwise we can write any testing script here.
• As my test case passed then next Job will be build (MergeBranches) and newDeveloper Branch will be merged to Master Branch.

• Now this Job 4 will be automatically build by which code in master branch will be deployed to web server and new output is accessible to client.

I am learning these skills in DevOps Assembly Lines Training Program organized by LinuxWorld, Jaipur under the mentorship of Mr. Vimal Daga sir.

Thank you very much for reading.