Building a CI/CD Pipeline with a Self-Hosted Runner and Cloudflare Tunnel

In today's fast-paced development environment, a robust CI/CD pipeline is crucial for seamless deployment and continuous integration. In this article, I'll walk you through how I built a comprehensive CI/CD pipeline using a combination of virtual machines, GitHub Actions, PM2, and Cloudflare Tunnel.

Step 1: Setting Up the Virtual Machine

To start, I used VMware to build a virtual machine (VM) running Ubuntu 24.04 Live Server. This VM served as the foundation for my development and deployment environment.

Step 2: Cloning the Repository

Once the VM was up and running, I cloned the project repository from GitHub using a Personal Access Token (PAT). This allowed me to access and manage the codebase on the VM securely.

Step 3: Exposing the VM with Cloudflare Tunnel

To make the VM accessible publicly, I used Cloudflare Tunnel. This tool provided a secure and reliable URL for accessing the VM, making it easy for external users to interact with my application.

Step 4: Configuring the Self-Hosted GitHub Actions Runner

Next, I installed and configured a self-hosted GitHub Actions runner on the VM. This runner allowed me to execute GitHub Actions workflows directly on my VM, facilitating automated builds and deployments.

Step 5: Setting Up GitHub Actions

With the runner in place, I configured GitHub Actions to handle the CI/CD process. This involved writing a .yml configuration file to automate tasks such as code checkout, dependency installation, building the client application, and restarting the server.

Step 6: Installing PM2 for Process Management

To ensure smooth operation of my applications, I installed PM2, a process manager for Node.js applications. PM2 facilitated automated process management, including restarting the server and handling application crashes.

Step 7: Testing and Verification

I tested the pipeline by pushing code changes to the repository. The CI/CD pipeline executed as expected, automatically building and deploying the application. This validated that the pipeline was functioning correctly.

Challenges Faced

Throughout the process, I encountered several challenges:

1. GitHub Actions Runner on Queue: Initially, the runner was stuck in a queue. This was resolved by ensuring the runner was properly configured and available.
2. Manual Runner Restarts: I faced issues with the runner needing manual restarts. To address this, I set up the runner as a service to ensure it remains connected and operational.
3. Repository Not Updating: There were instances where the repository was not updating. This was fixed by ensuring the runner had the correct permissions and configuration for accessing the repository.
4. VM Connectivity Errors: I encountered errors with the VM's internet connectivity. This was resolved by configuring network settings and ensuring the VM had proper internet access.
5. Switching Repository from Public to Private: Changing the repository from public to private caused authentication issues. I addressed this by updating the GitHub Actions configuration to use a PAT for authentication.

Building a CI/CD pipeline with a self-hosted runner and Cloudflare Tunnel was a rewarding experience. Despite the challenges, the setup automated deployment process, ensuring that code changes are seamlessly integrated and deployed.