Day 28- Jenkins Agents/90days of DevOps challenge

Jenkins is an open-source automation server that is widely used for continuous integration and continuous delivery (CI/CD) pipelines. It allows developers to automate various tasks such as building, testing, and deploying software applications.

In Jenkins, there are two main components: the Jenkins master and Jenkins agents (also known as Jenkins slaves). Let's take a closer look at each of them:

• Jenkins Master: The Jenkins master is the central server that manages and coordinates the entire Jenkins environment.
• It handles the scheduling of jobs, monitoring their execution, and distributing work to Jenkins agents.
• The Jenkins master is the heart of the Jenkins system. It is responsible for managing the entire Jenkins environment and coordinating the execution of jobs.
• It provides a web-based interface that allows users to configure Jenkins, create and manage jobs, and view the status of builds and pipelines.
• Jenkins Agents (Slaves):
• Jenkins agents are worker nodes that execute the actual build tasks. Agents are responsible for running jobs assigned to them by the master.
• They provide the necessary resources, such as computing power and tools, to perform the build or deployment tasks. Agents can be set up on different machines, including physical servers, virtual machines, or cloud instances.
• They communicate with the Jenkins master through a Java Network Launch Protocol (JNLP) or by connecting via SSH.
• Multiple agents can be connected to a single Jenkins master, enabling parallel execution of jobs and efficient utilization of resources.
• Agents can run builds concurrently, independently of each other, as long as the necessary resources are available.
• Jenkins also supports the concept of agent labels, which allow jobs to be associated with specific agent characteristics or capabilities. Labels help in selecting suitable agents for jobs based on predefined criteria.

A single, monolithic Jenkins installation can work great for a small team with a relatively small number of projects. As your needs grow, however, it often becomes necessary to scale up. Jenkins provides a way to do this called “master to agent connection.” Instead of serving the Jenkins UI and running build jobs all on a single system, you can provide Jenkins with agents to handle the execution of jobs while the master serves the Jenkins UI and acts as a control node.

When a job is triggered in Jenkins, the master selects an available agent based on certain criteria (such as labels or job requirements) and assigns the job to that agent. The agent then performs the build steps or executes the pipeline on its own environment. The master and agents communicate with each other to exchange information about the job status and progress.

Pre-requisites:

Let’s say we’re starting with a fresh Ubuntu 22.04 Linux installation.

To get an agent working make sure you install Java ( same version as Jenkins master server ) and Docker on it.

Task-1

Jenkins, a powerful automation server, can significantly boost its performance by distributing build workloads across multiple machines using a master-slave setup. This step-by-step guide is tailored for beginners and aims to help you seamlessly establish a Jenkins master-slave architecture, complete with the use of RSA keys for secure authentication. Let's dive in!

Create an agent by setting up a node on Jenkins

Create a new AWS EC2 Instance and connect it to the master(Where Jenkins is installed)

The connection of the master and agent requires SSH and the public-private key pair exchange.

Verify its status under the "Nodes" section.

Step1:

The first step is to create two EC2 instances and name them accordingly, I named them Jenkins_master and Jenkins_agent.

Now Go to the jenkins_master instance install docker and Jenkins , add $USER and Jenkins to docker group using

sudo usermod -aG docker $USER 
sudo usermod -aG docker jenkins         

Step 2:

Generate RSA Key Pair on Master Server

• Access the terminal on your Jenkins master server.
• Generate an RSA key pair: ssh-keygen
• Verify the key generation

ssh-keygen 
cd .ssh 
ls 
cat id_rsa.pub #public key        

Step 3:

• Add the Master Server's Public Key to the Slave Server
• Retrieve the public key from the master server: cat ~/.ssh/id_rsa.pub
• Navigate to the .ssh directory on the slave server: cd ~/.ssh
• Open the authorized_keys file using a text editor: nano authorized_keys
• Paste the master server's public key on a new line.
• Save the changes and exit the text editor.

Step 4: Connect to Slave from Master Server

Come to the Jenkins_master instance and try to connect to the Jenkins_agent via ssh.

Congratulations we have established a connection between two systems using SSH.

Now we can proceed with the next step, let's set up the agent on the master Jenkins server.

Step 1: Go to the Jenkins dashboard and set up an agent. ( if you don't see that go to manage Jenkins> Nodes ).

create a new node, name it (as you required) checkmark on the permanent agent, click on the create button, and it's created !!

Then on the next page, you will be asked to enter a few details such as description, remote directory, Host IP, and credentials

Configure the node settings:

• Set the desired number of executors (build threads).
• Define the remote root directory where Jenkins will work on the slave machine.
• Assign labels for categorizing nodes.
• Choose "Launch agent agents via SSH" as the launch method.
• Enter the "slave server's address" as the host.
• For credentials, choose "From the Jenkins master ~/.ssh" and select the private key associated with the public key added to the slave server.
• In Host Key Verifying Strategy select the option "Non verifying verification strategy".

Step 6: Launch the Agent

• Click "Launch agent" to initiate the connection between the master and slave.

Task-02

Run your previous Jobs (which you built on Day 26, and Day 27) on the new agent

Use labels for the agent, your master server should trigger builds for the agent server.

Step 7: Create and Run a Pipeline

• Create a new pipeline job in Jenkins and Write the pipeline script to define the stages and steps of your build process.

Save it and click on the build now button on the job's dashboard page.

• Ensure the job runs on the slave machine. Save the pipeline configuration and execute it.

Thank you for ??reading my blog, ??Like it and share it ?? with your friends.

Happy learning ???? !!!!