Dynamically scalable Jenkins slave with Docker and Kubernetes

Introduction

Jenkins is CI/CD tool with a long history and keeps evolving itself. It’s Master/agent architecture is great for scalability to do distributed builds. There are many ways to provision Jenkins Agent, from using bare-metal machines, Virtual Machines, dynamic EC2 instances, containers from Docker, or Kubernetes clusters.

The integration between Jenkins and Kubernetes cluster is great. With these benefits, I’ve fully migrated CI/CD pipeline from host(VM) based Agent to Pod based Agent.

• Dynamic Jenkins agent from Kubernetes, light weighted, provisioned on-demand within a few seconds
• Fresh and reproducible Jenkins agent environment for every build
• Resource/cost saving brought by Kubernetes

In this article, I’d like to share recent approach to dynamically provision a Jenkins Agent with simple lines of code using the method of Jenkins Shared Library.

In this agile world, to sustain in business we need to adopt the agile process or say DevOps environment. The future of DevOps is DevOps assembly line. To become agile we need to have a robust architecture system and many resources. The best way to manage is by using container technology.

Now few things which we can even do more better is to make it sustain the changing requirements of the customer. It's a very tedious and hard process to do updates in real life with zero downtime and even not disturbing the ongoing actions on the server. So for that Kubernetes provides us with a very prominent strategy to do updates or in the industry it's called roll out.

Even if we go more in-depth we can see that deployment is still very efficient but the process which we have in the production is still not that efficient. To even make it more scale-able we can have a concept of build configuration which will automatically fetch the code once the developer commits it to the SCM and make an image out of it. This image will be further uploaded to the public or private repository like docker hub.

Now the deployment process is the same, it will download from the docker hub or any other repository and deploy the code. Even if the new version or patch is to be implemented, the developer will only make changes and commit it again. Our build configuration will again make the image out of it and upload it. Now we will use a rolling update strategy to roll out because we have already made deployment. This whole process would be taking place on the dynamic slave of Jenkins which will be the image that we built and launched on top of Kubernetes.

How to create the Jenkins Dynamic Cluster?

For this, we have some pre-requisite to be done.

1. We need to update Jenkins to the latest version because we will install some plugins which need the newer version. To update run this code

yum update jenkins -y

Let it update then restart the services

systemctl restart jenkins

2. Now we need to download some plugins as mentioned bellow

1. SSH Agent
2. Docker
3. Yet Another Docker

manage jenkins -> manage plugins -> available -> search and download

3. Start the minikube which will in turn help us create kubernetes single node cluster as a server. For this, you require minikube which you can download from the internet. and start from command-line.

minikube start

4. Now in the docker internal service file we need to make some changes because we want to control it using Jenkins, hence we want to make Jenkins as a docker client and docker installed in a host as a server. Here we have to enable TCP connection as docker will accept it with the protocol TCP.

vim /usr/lib/systemd/system/docker.service

Here we appended this line which will allow to control docker server from anyone who contacts it

-H tcp://0.0.0.0:6969

Here you can give any port number.

5. Now we need to make come configuration in Jenkins so that it can dynamically launch cluster.

Go to manage jenkins -> Manage Nodes & Cloud -> Configure Clouds

Then choose docker from the dropbox appeared. Then do the following configuration

Here you will give localhost IP with the port number which we edited. in the section of docker image, I have hided Image because this is no longer available now.

Here you are seeing an error because some failure as to demonstrate this, I have stopped some services but you won't see this error. Here Remote File System is the path where you will have your all code files, so it's a good practice to make a directory, for me it's /jenkins-dynamic

Then click save. You wont see any node come up right now because its cloud, this means on demand or dynamic.

So, we have created our dynamic slave which will come up when its needed or called in use using labels. We will call it using labels, that's why it's import to give labels.

Jenkins Job Configuration

JOB slave-github

This will download code from the GitHub account we provide and then do build configuration.

After putting GitHub URL we can set a build trigger.

Now we are write script that is deployed docker container on the linux system.

Now let's see how our Dockerfile to create image of our code and configure it as kubernetes client should be.

FROM centos
RUN yum install sudo -y 
RUN yum install java -y 
RUN yum install openssh-server -y 
RUN /usr/sbin/sshd -D & 
RUN ssh-keygen -A 
RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl &&\
RUN chmod +x ./kubectl 
RUN mkdir /root/.kube /root/jenkins 
RUN sudo mv ./kubectl /usr/local/bin/kubectl 
COPY ca.crt client.crt client.key config deployment.yml /root/
RUN mv /root/config /root/.kube/config &&\ 
    yum install httpd -y &&\
    yum install php -y 
COPY index.html /var/www/html/
EXPOSE 80
CMD /usr/sbin/httpd -DFOREGROUND

For this you have to take your keys and certificates from host system to RHELv8 machine and keep in the same directory as the Dockerfile

This is after running this jobs that's why you are seeing index.html and README.md files. You don't have to create this.

apiVersion: v1
kind: Config
clusters:
- cluster:
    server: https://192.168.99.100:8443
    certificate-authority: /root/ca.crt
  name: minikube
contexts:
- context:
    cluster: minikube
    user: shubham
users:
- name: shubham
  user:
    client-key: /root/client.key
    
    client-certificate: /root/client.crt

This is the config file. This will configure the slave node to communicate to kubernetes server.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myweb
spec:
  replicas: 2
  selector:
    matchLabels:
      env: slave
  template:
    metadata:
      name: myweb-pod
      labels:
        env: slave
    spec:
      containers:
      - name: myweb-con
        
        image: shubhambhalala/slave

This is the deployment.yml file which will deploy our application on top of dynamic jenkins.

JOB slave-deployment

This job will run on a dynamic jenkins node and will deploy our application on top of Kubernetes cluster.

Now if the deployment is already created then it will do rolling update for the new version on fly. This will be done using rolling strategy hence there will be no downtime or latency.

Here we have to specify the label we set for the slave node. NOTE: This option will only come after you have configured the slave node, until then you won't see this option.

Now we are running a bash script which gonna help to update our environment on the fly.

This will finally launch our application and expose it so that the client can have access to this.

Finally, we will create a build pipeline which is something that show us the flow of Jenkins jobs.

You can see in console output that how it does everything. Let's see what happens in jobs output.

Now see image built successfully using Dockerfile

Downloaded necessary software

Now let's see for the job2

You will see this if you are running it for the first time. Next time you change something for update and run it, you will see its doing rolling update.

Finally, if you can take the port number and view your website deployed using dynamic Jenkins cluster on Kubernetes with rolling update. you can see your website is live and updated.

Thank you

Hope you liked it!!

if Any query please dm me and also suggest something.