Integration of CI/CD Pipeline with Dynamic Distributed Clusters
Ashutosh Kumar Sah
DevOps Engineer @CoffeeBeans | Ex - Kredifi | Ex - Teqfocus | Microsoft Azure Certified: Az-900, Ai -900, Dp-900 | Oracle cloud infrastructure certified fundamental 2022 | Aviatrix certified DevOps cloud engineer |
We are going to create a project given as a task of Devops Training under the mentorship of Mr. Vimal Daga
Task is:
Create A dynamic Jenkins cluster and perform task-3 using the dynamic Jenkins cluster.
Steps to proceed as:
1. Create container image that’s has Linux and other basic configuration required to run Slave for Jenkins. ( example here we require kubectl to be configured )
2. When we launch the job it should automatically starts job on slave based on the label provided for dynamic approach.
3. Create a job chain of job1 & job2 using build pipeline plugin in Jenkins
4. Job1 : Pull the Github repo automatically when some developers push repo to Github and perform the following operations as:
1. Create the new image dynamically for the application and copy the application code into that corresponding docker image
2. Push that image to the docker hub (Public repository)
( Github code contain the application code and Dockerfile to create a new image )
5. Job2 ( Should be run on the dynamic slave of Jenkins configured with Kubernetes kubectl command): Launch the application on the top of Kubernetes cluster performing following operations:
1. If launching first time then create a deployment of the pod using the image created in the previous job. Else if deployment already exists then do rollout of the existing pod making zero downtime for the user.
2. If Application created first time, then Expose the application. Else don’t expose it.
Now we are going to create the task
A CI/CD Pipeline implementation, or Continuous Integration/Continuous Deployment, is the backbone of the modern DevOps environment. It bridges the gap between development and operations teams by automating the building, testing, and deployment of applications.
But we face one more problem i.e less compute unit. It can be solved cloud computing but in this project, we will discuss another way. So, we will be using the concept of Dynamic Distributed Jenkins Clusters.
This project will help to deploy a website using a Dynamic Distributed Jenkins Cluster on Kubernetes using a remote Slave Node. it is the main goal for this project.
Everything is configured in such a way that at the moment developer will commit the Dockerfile and webUI code on GitHub, the slave will launch the Distributed cluster and the image will be built and pushed to Dockerhub and the Deployment will be started on K8s.
If the developer updates the code and recommits it, the Cluster will use its Rolling Updates feature to update the Website and the website will be uploaded with zero Down Time.
For understanding the project, you need some prior knowledge.
Note: Please, view the article in browser for better view of images.
This is our Kubernetes Dockerfile for creating image
For building this image we will use the following files
- ca.crt
- client.crt
- client.key
- config
- Dockerfile
- kube.repo
this is our kube.repo file which we are using in the project
config
docker build -t kube:v1 /k8s/
this is our WebUI Dockerfile :
Now we are going to the process of Configuring Docker Service
We need to configure Docker Service from the localhost to work as a Client and not as a Server. We need to set this to use docker remotely. To configure edit the file /usr/lib/systemd/system/docker.service as follows:
Use this Command after configuring the file
systemctl daemon-reload systemctl restart docker export DOCKER_HOST=<ip_address>:2102
Now we are going to Configure Cloud in Jenkins
Docker plugin must be installed before doing this step :-)
At first we have to go to Manage Jenkins then manage nodes and clouds nd then configure clouds and then add a new cloud then docker
after this we will do some steps more as shown in next figure:
it is not completed yet so wait for the next image also
Now we have completed the Setup.
Job1: Build and Push Image
After giving our Github url in Github project section we have to do is : go to the buid trigger section:
and then we move forward and go to the Execute Shell and write some scripts for copying the file into the mounted volume.
After this job is successfully built, it will trigger job2
Job2: Deployment and Rolling Update
This was the complete architecture. Deployments will launch the pod again as soon as the pod goes down
MY GITHUB link is : https://github.com/ashu-cybertron/devops-task-4.git
This all happend due to only the person best mentor Mr. Vimal Daga sir.
DevOps, Cloud & Performance Engineer | MTS-2 @Cohesity
4 年Inspiring