DEVOPS AUTOMATION
Nikhil G R
Senior Data Engineer (Apache Spark Developer) @ SAP Labs India, Ex TCS, 3x Microsoft Azure Cloud Certified, Python, Pyspark, Azure Databricks, SAP BDC, Datasphere, ADLs, Azure Data factory, MySQL, Delta Lake
This is the structure of the task.
Here we assume that you have installed Jenkins, Docker, and have git repo.
We have made three jobs JOB1, JOB2, JOB3.
Each job has its work so let's go stepwise and deploy the project.
Here lets assume the main developer as a master branch and developer1 as branch of master branch.
JOB1
JOB1 is fetching data from developer1 and launching it to a new container (Testing) through defined route, so that the Quality checking team can check the quality and verify the code and decide whether it is good to go or not.
STEPS TO MAKE JOB1:-
STEP1- Click on New item.
STEP2:- Enter the item name(It means a job name) and then select freestyle project and then Click OK.
STEP3:- Open configure option by clicking on its file and do the following changes in it.
IN REPOSITORY URL :There will be URL of your repo from which you want data to come.
IN BRANCH SPECIFIER: There will be branch fo developer1.
Now we have to build a trigger, here we are using "pole SCM".
Now we have to write command to execute "shell".
Commands to be given are as under, which can be copied and pasted. GitHub link is also provided to locate that file.
sudo cp -v -r -f * /testing #!/bin/bash if sudo docker ps | grep main os then echo "already running"
else sudo docker run --name mainos -dit -p 8082:80 -v/mainenv:/usr/local/apache2/htdocs/ httpd fi
After this click on the save button and check the output.
We are done with JOB1. Its time to check the output.
This is git repo from which data is been fetched by Jenkins.
This is the output from testing env this text is fetched from my dev1 branch.
So here our Firt job (JOB1) is complete. It will launch a new test container, that can be accessed by the quality checking team and if the container is already launched it will write the container already running.
Moving to Making job2
Steps to make JOB2:-
Step1:- Click on New Item.
Step2: Enter the item name. Here item name means job name. Select freestyle project and Click OK.
STEP3:- Open the configure option by clicking on it and do the following changes in it.
After opening the configure file, do the following changes in it.
IN REPOSITORY URL :There will be URL of your repo from which you want data to come.
IN BRANCH SPECIFIER:There will be branch of maindevp(master branch).
Now a trigger is to be built, here "pole SCM" is used.
Now we have to write command to execute "shell".
Commands to be given are as under, which can be copied and pasted. GitHub link is also provided to locate that file.
sudo cp -v -r -f * /mainenv/ #!/bin/bash if sudo docker ps | grep mainos then echo "already running" else sudo docker run --name mainos -dit -p 8082:80 -v /mainenv:/usr/local/apache2/htdocs/ httpd fi
After this click on the save button and check the output.
Now we are done with JOB2. Its time to check the output.
This is git repo (master branch) file data.
This is the main environment output which is available to the client.
Here our second job (JOB2) is completed. It will launch a new "main container" that can be easily accessed by the client and if the container is already launched it will write the container already running.
So till here we have already automated two jobs and now the last job which is a little bit tricky. So let's proceed toward it.
Steps to make JOB3:-
Step1:-Click on New Item.
Step2: Enter item name, here item name is equal to job name and select freestyle project and click OK.
STEP3:- Open the configure option by clicking on it and do the following changes in it.
After opening the configure file, do the following changes in it.
IN REPOSITORY URL :There will be URL of your repo from which you want data to come.
IN BRANCH SPECIFIER:There will be branch of main developer(master branch).
Here remember to add credentials of the GitHub
You can get an additional behavior panel by clicking on it and from it select "merge before build" and follow the below method.
In Job 3, we are making a trigger token, so that it can be run by only the Quality management team. If the work passes all the quality norms, the Quality management team will run a token to merge the work with the main environment. In case work fails the Quality norms, Quality management team will send a message to "developer 1" to rectify the defect.
To generate token, here is the line
curl --user "username of Jenkin: password of Jenkins" Jenkinsurl/job/merging/build?token=redhat example:
curl --user "admin: root" https://192.168.0.106:8080/job/merging/build?token=redhat
This trigger can only be run by a quality team to run a job.
Now we have to write command to execute "shell".
Command to be given are as follows, which can be copied and pasted. GitHub link is also provided to locate that file.
Here in Job3 if the testing container is running, it has to be deleted and if it's not there, a message is to be printed.
#!/bin/bash if sudo docker ps | grep testenv then sudo docker rm -f testenv else echo "container removed"
fi
After this, select "post-build" option and do the following changes in it.
project to build = job2 name
As we need to merge test env into main and upload it on the main environment, make sure that rest changes are same as it is.
After this click on the save button.
We are done with JOB3. Its time to check the output.
Token is being run by a quality team.
Here is the final output.
Below we can see our testing env is destroyed.
So basically our JOB3 will merge two git branches and update it to the main environment and also destroy the testing environment as soon as the given URL is run by the quality team.
SUMMARY
In this whole project, we have automated system such that if "developer1" do some upgrade in the main developer( in the master branch), that work is first tested in the testing environment and it is checked by the quality management team and if work passes all quality norms, a token will be triggered by Quality management team, that will trigger JOB3 so that it can merge two branches and update the main branch to the main environment and destroy the testing environment.
Here is GitHub links for all commands
Thanks guys for reading this article. Hope you guys have enjoyed it.
Engineer
4 年Great work NIKHIL GR ??