AWS Project: Setting Up a Node.js App on EC2 Instance with Private mysql RDS and CloudWatch Logs

I recently worked on an AWS project where I deployed a Node.js application on an EC2 instance, connected it to a private RDS instance, and configured CloudWatch to monitor application logs. It was a hands-on experience that taught me a lot about securing AWS resources and setting up monitoring for better visibility.

In this post, I’ll walk you through the steps I followed, some challenges I faced, and the key takeaways from the project.

I am not including simple steps like create EC2 instance and how to connect to SSH.

Step 1: Clone the Repository

The first step was to clone the project repository to the EC2 instance. This ensured that I had the latest version of the code ready to deploy:

Step 2: Install Node.js and npm

Since the app was built using Node.js, I installed Node.js (version 20) and npm using the following commands:

curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - 
sudo apt install -y nodejs        

To verify the installation:

node -v
npm -v        

Step 3: Configure a Private RDS

I set up an AWS RDS (MySQL) instance in a private subnet to ensure it wasn’t directly exposed to the internet.

Configured RDS Security Group to accept trafiic from EC2 Security group only.

To connect to it securely from the EC2 instance, I installed the MySQL client.

sudo apt install mysql-client -y        

Connected to the RDS server:

mysql -h saurabhbhargav-db1.cvw4e48yi76b.ap-south-1.rds.amazonaws.com -u admin -p        

? Creating a Database and Table:

Once connected, I created a new database and a Users table:

CREATE DATABASE TestDB;
USE TestDB;
 CREATE TABLE IF NOT EXISTS Users ( ID INT PRIMARY KEY AUTO_INCREMENT, Name VARCHAR(50) NOT NULL, Email VARCHAR(50) NOT NULL );        

? Insight: Keeping the RDS instance in a private subnet reduces the attack surface and improves security.

Step 4: Configure pm2 to run application

Installing pm2 on ec2 so it can manage application autorun, application logs.

sudo npm install -g pm2        

Installed node modules in server

npm i        

pm2 start "node app.js" --name mysql-app1        

You can check logs using

pm2 logs mysql-app1        

Application is working on port 5000 : Make sure you allow 5000 port from EC2 Instance

?? Step 5: Configure Logs with CloudWatch

To monitor the application and keep track of errors, I decided to set up CloudWatch Logs. First, I installed the CloudWatch Agent:

sudo apt update 
sudo apt install -y wget 

wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb 

sudo dpkg -i -E amazon-cloudwatch-agent.deb        

? Creating a CloudWatch Configuration:

I created a configuration file for CloudWatch at /opt/aws/amazon-cloudwatch-agent/bin/config.json:

{
  "logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/home/ubuntu/.pm2/logs/mysql-app1-out.log",
            "log_group_name": "pm2-out-logs",
            "log_stream_name": "{instance_id}"
          },
          {
            "file_path": "/home/ubuntu/.pm2/logs/mysql-app1-out.log",
            "log_group_name": "pm2-error-logs",
            "log_stream_name": "{instance_id}"
          }
        ]
      }
    }
  }
}        

? Tip: Make sure the log paths and permissions are correct to avoid configuration errors.

?? Step 6: Add IAM Roles for CloudWatch Access

To allow CloudWatch Agent to push logs, I attached the following roles to the EC2 instance: ?? CloudWatchAgentServerPolicy

?? AmazonSSMManagedInstanceCore

?? Step 7: Start and Enable CloudWatch Agent

Finally, I started the CloudWatch Agent using this command:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \ -a fetch-config \ -m ec2 \ -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json \ -s        

Enabled CloudWatch Agent to start on boot:

sudo systemctl enable amazon-cloudwatch-agent 
sudo systemctl status amazon-cloudwatch-agent        

? Insight: This ensures that the agent starts automatically after a system reboot.

?? Challenges Faced

• Database connectivity: Initially, I faced issues connecting to the RDS instance because the security group didn’t allow inbound connections from the EC2 instance. After adjusting the security group rules, the issue was resolved.
• Log permissions: CloudWatch was not collecting logs initially because the log file permissions were not set correctly. Adjusting the file permissions fixed the issue.

? Outcome

?? Successfully deployed a Node.js app on EC2

?? Connected securely to a private RDS instance

?? Configured CloudWatch to monitor logs for better visibility and faster debugging

?? Key Takeaways

?? Always place RDS instances in a private subnet for better security.

?? CloudWatch monitoring helps to quickly identify and resolve issues.

?? Using IAM roles simplifies the configuration and reduces the need for hard-coded credentials.