Mastering CI/CD with AWS EC2: Your Roadmap to Seamless Development and Deployment

In today's article, I'm excited to guide you through the process of setting up a powerful and efficient Continuous Integration and Continuous Deployment (CI/CD) pipeline on AWS EC2 with Ubuntu. We'll start by creating an Ubuntu server on Amazon's Elastic Compute Cloud (EC2) and proceed to install Jenkins, configuring it for optimal performance. Along the way, I'll demonstrate how to access Jenkins and install essential plugins to enhance its functionality.

But the journey doesn't stop there! We'll delve into the world of CI/CD, where we'll create a seamless pipeline for your projects. We'll show you how to establish a GitHub repository and push your application or project onto it. The magic happens when we automate the entire process, ensuring that any changes made in your repository trigger the pipeline, enabling code to run smoothly, efficiently, and effortlessly.

By the end of this article, you'll be well-equipped to streamline your development workflow, ensuring that your projects are always up-to-date and ready to deploy. So, let's get started on your journey toward CI/CD excellence!

?? Navigate to the AWS Management Console, and from there, locate and note the public IP address associated with your server or machine.

→ Go to the AWS console and in the EC2 section click on " Launch Instance".

→ Enter the name of the server/machine (I putt Jenkins):

→ Attach key pair :

→ Allow two inbound rules. The first type is SSH and the second type is All Traffic. Set source type as Anywhere of both.

→ Click on Launch instance:

→ Instance has been launched successfully:

→ Now connect to the instance using SSH client:

→ I have login my machine/instance using git bash, first, I granted chmod permissions and then used the SSH command to log in:

→ Now I use the sudo su command in Ubuntu is used to switch the current user context to the root user (superuser) or any other user account, typically while maintaining elevated privileges.

Command: $ sudo su

→ No I run "sudo apt update" command to ensure that the package manager has up-to-date information about the software packages available in the repositories. This is important because it allows me to make informed decisions when installing or upgrading software, ensuring that you're using the latest versions and benefiting from security patches and bug fixes. It's a recommended step to take before using sudo apt upgrade or sudo apt dist-upgrade to update the installed packages on your system.

Command: $ sudo apt update

?? Install Java

→ Now I am using "sudo apt-cache search openjdk" comman to receive a list of packages that match the search term "openjdk." These packages may include various versions and components of OpenJDK, allowing me to identify and choose the specific OpenJDK packages I want to install on my Ubuntu system. This can be helpful when you're looking for a particular version of OpenJDK or specific components like the JRE (Java Runtime Environment) or JDK (Java Development Kit).

Command: $ sudo apt-cache search openjdk

→ I'm running "sudo apt install openjdk-11-jre" the will download and install the OpenJDK 11 JRE package along with its dependencies from the official Ubuntu repositories. Once the installation is complete, I'll have a functioning Java runtime environment on my system, allowing me to run Java applications, execute Java-based programs, and work with Java applications that require the JRE.

Command: $ sudo apt install openjdk-11-jre

→ I'm running "java -version" command to display the version information of the Java Runtime Environment (JRE) or the Java Development Kit (JDK) installed on my system. When I run this command, it provides information about the currently configured Java runtime environment, including the version number, build information, and sometimes additional details.

Command: $ java -version

→ No I run "sudo apt update" command after installing a package is a best practice to ensure that your system's package cache is synchronized with the latest information from repositories, helping you maintain an up-to-date and well-managed system

Command: $ sudo apt update

?? Install Jenkins

→ I'm running the following command that downloads the Jenkins GPG key, and with superuser privileges, it saves the key in the /usr/share/keyrings/ directory with the filename jenkins-keyring.asc. The GPG key is typically used for verifying the authenticity of software packages and repositories, which is crucial for package management and security. The > /dev/null part at the end is used to suppress any output that would otherwise be displayed on the terminal during the execution of the command.

Command: curl -fsSL https://pkg.jenkins.io/debian/jenkins.io.key | sudo tee \ /usr/share/keyrings/jenkins-keyring.asc > /dev/null

→ I'm running the following command to add a new APT repository source for the Jenkins Debian packages, specifying the repository URL and the location of the GPG key for package verification. This allows APT to access and manage Jenkins packages with the necessary security checks. The > /dev/null part at the end is used to suppress any output that would otherwise be displayed on the terminal during the execution of the command.

Command: echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \ https://pkg.jenkins.io/debian binary/ | sudo tee \ /etc/apt/sources.list.d/jenkins.list > /dev/null

→ Now I run "sudo apt update" command which is essential after adding a new repository source to make sure that your system is aware of the latest available packages and their versions, which is important for package management and system maintenance.

Command: $ sudo apt update

→ I'm running "sudo apt-get install jenkins" that will download and install the Jenkins package along with its dependencies from the official Ubuntu repositories (assuming that Jenkins is available in the configured repositories). After the installation is complete, Jenkins is set up as a system service, and I can start, stop, and configure it to automate various tasks, such as building, testing, and deploying applications, and managing software development processes.

Command: $ sudo apt-get install jenkins

→ Now run "sudo systemctl enable jenkins" command which configures systemd to ensure that Jenkins starts automatically during system boot or initialization. This is essential for services like Jenkins, which need to be available and running without manual intervention to perform tasks such as continuous integration and continuous delivery (CI/CD) jobs.

Enabling the Jenkins service in this way ensures that it will start whenever the system is powered on or restarted, making it readily available for its intended tasks.

Command: $ sudo systemctl enable jenkins

→ Now run "sudo systemctl start jenkins" command which is initiating the immediate start of the Jenkins service. This is useful when I want to start the service right away without waiting for the next system boot. It allows me to manually control the Jenkins service and put it into an active state so that it can perform its tasks, such as running build jobs, deploying applications, and handling automation processes.

Starting the Jenkins service manually is helpful when i need it to perform specific tasks or when I want to check its status and troubleshoot any issues that may arise during its operation.

Command: $ sudo systemctl start jenkins

→ Now I run the "sudo systemctl status jenkins" command is a powerful tool for monitoring the health and operation of system services, such as Jenkins. It provides insight into the service's status, resource usage, and any recent events or log messages related to its operation. This command is valuable for system administrators and users responsible for managing and maintaining services on a Linux (Ubuntu) system.

Command: $ sudo systemctl status jenkins

?? Access and setup Jenkins

→ Now I'm going to my AWS console and copy the public IP of my Ubuntu server.

→ Now On my local computer, I'm opening a web browser (e.g., Chrome, Firefox, or Safari).

In the address bar of my web browser, I enter the Public IP of my Ubuntu server/machine, including the port number (8080). For example, my Ubuntu server's IP address is 35.76.108.58, so I enter:

https://35.76.108.58:8080 or 35.76.108.58:8080

→ As you know I'm accessing Jenkins for the first time, and I'm prompted to unlock it. I see a screen with the location of the initial "Administrator password." This password can be found on the server in the Jenkins home directory in a file named secrets/initialAdminPassword.

To retrieve the password, go to the terminal and run a command like:

$ cat /var/lib/jenkins/secrets/initialAdminPassword

→ Copy the password and paste it into the Jenkins web interface then click on continue:

→ Click on Install suggested plugins:

→ After installing the plugin I am referred to a screen where I create the first admin user, after putting details click on save and continue:

→ On instance configuration step, if you want to change Jenkins URL then you will use the private IP of your Ubuntu server but here I am not changing and click on save and finish:

→ Congratulations! Jenkins has been setup successfully:

→ After clicking start using Jenkins I am entered into the dashboard of Jenkins:

??Creating Jenkins CICD pipeline with GitHub

→ Create a new job, enter an item name and select freestyle project then click OK.

→ Enter description and add Github repository URL.

→ Get URL form github public repository (you can use my code).

→ Now in source code management section on Jenkins, select Git as source code management then click to save.

→ Now click on Build now to check whether our job works or not.

→ We can see in the Console Output section that our job builds successfully.

→ Go to build workspace path (/var/lib/jenkins/workspace/todo-node-app) in terminal by running following command and run the commands one by one which are given in readme file of github repository

→ Goto to path:

Command: $ /var/lib/jenkins/workspace/todo-node-app

→ Run the following commands

? Command: $ sudo apt install nodejs

This command is used to install the Node.js runtime environment on your Ubuntu system.

? Command: $ sudo apt install npm

This command is used to install the Node Package Manager (npm) on an Ubuntu system.

? Command: $ npm install

This command is used to install dependencies for a Node.js project. When you run this command within a Node.js project directory, npm will read the project's package.json file and install all the packages and dependencies listed in it.

? Command: $ node app.js

This command is used to run a Node.js application

After this, go to the AWS console and copy the public IP of the instance then use: the 8000 port to access the application.

Now it's clear that our application runs successfully, but if I press ctrl + C, my application stops but I need to make dockerfile that automates this and doesn't stop the application.

As you know I run commands manually from installing node js to run application but now I'm going to automate these commands via docker.

→ Create a file named Dockerfile using following command:

? Command: $ vi Dockerfile

then I write the following script that will do everything that I have done manually:

? Script:

FROM node:18-alpine

WORKDIR /app

COPY . .

RUN npm install

RUN npm run test

EXPOSE 8000

CMD ["node","app.js"]

and exit file (as you know how to save and exit vi file).

→ Before going to building a docker image i need to Install docker in ubutu:

? Command: $ apt install docker.io

→ Now the docker file has been created so I'm going to use the following command to build a Docker image from a Dockerfile and tag it with a specific name:

? Command: $ docker build . -t node-app

The docker image has been built successfully.

→ Now I use the following command to run a Docker container based on an image called node-app:

? Command: $ docker run -d --name node-todo-app -p 8000:8000 node-app

→ Now I use the following command to list running Docker containers on your system. It provides a summary of information about these containers, allowing you to see their status, resource usage, and other relevant details.

? Command: $ docker ps

→After this I integrated the GitHub code using docker. I go to my browser and in the search bar I write my public IP with my port number and search:

→ Now I am going to run my code through jenkins:

? Before proceeding, first kill the docker container using the following command:

? Command: $ docker kill <container id>

? I use the following command to change the permissions of the directory /var/lib/jenkins/workspace/todo-node-app in a way that gives everyone full control over the directory and its contents

? Command: $ sudo chmod 777 /var/lib/jenkins/workspace/todo-node-app

? Use the following command to add Jenkins to the Docker group. Replace "jenkins" with the actual Jenkins username if it's different in your environment.

? Command: $ sudo usermod -a -G docker jenkins

? After adding Jenkins to the Docker group, restart the Jenkins service for the changes to take effect.

? Command: $ sudo systemctl restart jenkins

? Now go to jenkins dashboard > job>todo-nodeapp ((your job name) > configure > Build Environment

Then go to "Build Steps" select "Execute shell" and write that we run mannually for building and running image:

? Script:

docker build . -t node-app-todo

docker run -d --name node-app-container -p 8000:8000 node-app

→ After clicking on built now our job, the build became successful and application started:

→ Now I need to make ci/cd pipeline that runs automatically, whenever anyone pushes anything in the code GitHub repo:

? Kill running container

? Install a plugin in jenkins named: Github Integration

? Now go to the settings of the Github repository > webhooks > Add webhook

? In the payload section enter the payload of Jenkins:

like: https://3.112.91.121:8080/github-webhook/

and select Content type as application/json then click add webhook:

? Set the inbound rule for port 8080 and select source as Anywhere.

? Now go to jenkins dashboard > job>todo-nodeapp ((your job name) > configure > Build Triggers and select "GitHub hook trigger for GITScm polling" and click save.

?? Final Testing

→ Now go to the GitHub repository and make any change in any file/commit anything:

→ If you face a container conflict issue then run the following commands or change the container name in Build steps if not then skip this step:

? Command: $ docker stop node-app-container

? Command: $ docker rm node-app-container

→ After commit, my ci/cd pipeline starts automatically:

Finally, our Jenkins CICD with GitHub Integration has been created??.

Stay on top of your career game by following me,?Muhammad Abdullah, and activating notifications on my profile by clicking the (??) button.

Don't keep them to yourself – let's get talking in the comments below!

Happy Learning!

?? Enjoying my content? Stay in the loop! ????

Follow me here:?Muhammad Abdullah

Connect with me on:

??WhatsApp: +923374840228 /?wa.me/+923374840228

?? Instagram for engaging visuals:?https://lnkd.in/dgapY6CP

Github repository Link: https://github.com/TheAbdullahChaudhary/node-todo-cicd-project