Setup Dynamic Docker Worker Agent

Problem Statement:

• Jenkins jobs are running based on labels which are specific to the nodes which leads to not using node at its fullest.
• This is because of when one job is running other jobs on the same node goes on waiting.
• When there are no jobs, nodes are sitting idle and cannot be used.
• There would be too many nodes which would be required in this case which will increase the cost and maintenance since the nodes would be configuration specific or OS dependent.
• Jobs won’t be running in isolation which sometime creates problem since one config could affect other.

Solution:

Dynamically creating nodes and running automation on that solves the problem in case of Jenkins. There are multiple approaches to the same and each one of them have their own advantages and disadvantages.

Approach 1:

Approach 1.1

Approach 1.2

?Approach 1 shortcomings:

• Maintenance of Docker files and Jenkins files
• Modification is needed for all repo in Jenkins file.
• All the nodes are needed to connect to master all the time (maintenance if the communication is broken or service goes down etc.)
• Not using node fully
• Parallel execution of jobs is not possible in this scenario.
• Windows Node would be paid for Docker.

Approach 2:

Approach 2 benefits:

• Utilizing the node fully
• Parallel execution is possible .
• Centralized docker solution, maintenance is easy .
• Jenkins file modification is not required .
• Repo specific Docker Files are not required .
• Upgrading to versions for tools and utilities is easy.

Approach 2 Shortcoming:

• Central docker images should be generic (we need to create different docker images for API, Web etc)
• Need to have separate and dedicated node which doesn’t support docker or explicitly need separate server (Need to use approach 1 or normal approach which we are following now)

Approach 2 Implementation:

Configure Docker Host With Remote API:

Jenkins master connects to the docker host using REST APIs. So we need to enable the remote API for our docker host.

Make sure the following ports are enabled in your server firewall to accept connections from Jenkins master.

32768 to 60999?is used by Docker to assign a host port for Jenkins to connect to the container. Without this connection, the build slave would go in a pending state.

Lets get started,

Step 1:?Spin up a VM, and install docker on it. You can follow the official?documentation for installing docker.?based on the Linux distribution you use. Make sure the docker service is up and running.

Step 2: Log in to the server and open the docker service file?/lib/systemd/system/docker.service. Search for?ExecStart?and replace that line with the following.

ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock        

Step 3:?Reload and restart docker service.

sudo systemctl daemon-reload
sudo service docker restart        

Step 4: Validate API by executing the following curl commands. Replace?54.221.134.7?with your host IP.

curl https://localhost:4243/version        

Create Docker file:

FROM ubuntu:18.0

# Make sure the package repository is up to date.
RUN apt-get update && \
? ? apt-get -qy full-upgrade && \
? ? apt-get install -qy git && \
# Install a basic SSH server
? ? apt-get install -qy openssh-server && \
? ? sed -i 's|session? ? required? ? ?pam_loginuid.so|session? ? optional? ? ?pam_loginuid.so|g' /etc/pam.d/sshd && \
? ? mkdir -p /var/run/sshd && \
# Install JDK 8 (latest stable edition at 2019-04-01)
? ? apt-get install -qy openjdk-8-jdk && \
# Install maven
? ? apt-get install -qy maven && \
# Cleanup old packages
? ? apt-get -qy autoremove && \
# Add user jenkins to the image
? ? adduser --quiet jenkins && \
# Set password for the jenkins user (you may want to alter this).
? ? echo "jenkins:jenkins" | chpasswd && \
? ? mkdir /home/jenkins/.m2
COPY .ssh/authorized_keys /home/jenkins/.ssh/authorized_keys

RUN chown -R jenkins:jenkins /home/jenkins/.m2/ && \
? ? chown -R jenkins:jenkins /home/jenkins/.ssh/

# Standard SSH port
EXPOSE 22

CMD ["/usr/sbin/sshd", "-D"]4        

Prepare Jenkins to configure docker cloud and template:

Cloud configuration:

Go to Jenkins -> Manage Jenkins -> Manage Cloud -> Add cloud

Configure the agent template by providing following details

Utilize Docker cloud in Jenkins Job:

Use docker node label while configuring a jenkins job, use the same template which has been configured in the docker template while configuring the docker cloud

This to be affected and needs Testing:

• Video recording of the feature has been tested thoroughly.
• Screenshot capturing feature needs testing.
• Resolution specific things will be impacted.
• If there are any specific testing happens like DB, those drivers need to be tested and taken care while writing down docker file