Let’s Talk About Observability in AWS ??

Hey AWS enthusiasts! Today, we’re diving into something super interesting and absolutely essential—Monitoring. Think of it as giving your systems a smartwatch that tracks their health, performance, and well-being!

Why Does Monitoring Matter? ??

Imagine this: your AWS system is a finely tuned orchestra. Without monitoring, you wouldn’t know if the drummer (CPU) is skipping a beat or if the violinist (memory) is taking an unscheduled coffee break. Monitoring helps you:

1. Keep track of system/workload performance ??
2. Ensure resources are used wisely (no overachieving CPUs, please!) ?
3. Check if systems are in good health ??
4. Maintain tight security ??

And now, let’s spotlight the superhero of AWS monitoring: Amazon CloudWatch!

Introducing Amazon CloudWatch: The Monitoring Maestro ????

CloudWatch is AWS’s all-in-one Monitoring and Observability Service. It collects data (logs, metrics, and events) and turns it into insights with stunning dashboards. Whether your applications run on AWS or on-premises, CloudWatch keeps you in the loop like a trusted sidekick.

In technical terms : CloudWatch collects monitoring and operational data in the form of logs, metrics, and events, and visualizes it using automated dashboard so you can see the unified view of AWS resources, applications, and services that run On AWS and on Premises.

What Makes CloudWatch So Cool? ??

1. Metrics: These are like fitness stats for your systems. Example: CPU Utilization—because who doesn’t want a fit EC2 instance?
2. Events: Real-time notifications of what’s happening. Example: “Oops, an EC2 instance just terminated!”
3. Alarms: Your system’s wake-up call when things go south. Example: “Email the admin if CPU usage goes berserk.”

A Tale of Two Metrics: Standard vs. Custom ??

• Standard Metrics: The basics, updated every 5 minutes. Think: “Hey, how’s my CPU?”
• Custom Metrics: The extra spicy details you set up yourself. Example: “How much memory is this EC2 instance hogging?”

Standard Metrics are enabled by default, and to setup the custom metrics, we need to follow the below steps!

STEPS TO FOLLOW:

?

1. Install the CloudWatch agent in the resources e.g., Ec2 Instances
2. Assign Role to EC2 to write metrics to the CloudWatch
3. Alarm will be triggered if the memory utilization beyond the setup threshold
4. Email Notification to the user

Hands-On Adventure: Setting Up Custom Metrics ???

Ready to roll up your sleeves? Here’s your guide to CloudWatch mastery:

Step 1: Spin up an EC2 instance.

Use the following user data script to install Perl and download the CloudWatch Monitoring agent Script.

#!/bin/bash

yum update -y

sudo yum install -y perl-Switch perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https perl-Digest-SHA.x86_64

cd /home/ec2-user/

curl https://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.2.zip -O

unzip CloudWatchMonitoringScripts-1.2.2.zip

rm -rf CloudWatchMonitoringScripts-1.2.2.zip

Note: For details on creating an EC2 instance with user data, refer this article - https://www.dhirubhai.net/pulse/creating-ec2-instance-user-data-simple-website-guide-ravikumar-fegrc/

Step 2: Enable Detailed Monitoring:

Once the EC2 instance is running, click Manage Detailed Monitoring to enable it.

Important: Detailed monitoring incurs costs and is not included in the AWS Free Tier. Be cautious!

Step 3: Verify the User Data Execution.

Log in to the instance via SSH or EC2 Instance Connect and check whether the user data script executed successfully.

Step 4: View Basic Metrics.

By default, basic monitoring (e.g., CPU Utilization, Network In/Out) is available in the Monitoring section of the EC2 dashboard.

Step 5: Attach an IAM Role.

To allow the EC2 instance to send custom metrics to CloudWatch, attach an IAM Role with the necessary permissions.

1. Navigate to IAM → Roles → Create Role.
2. Select AWS Service and attach a policy such as CloudwatchEventsFullAccess.

Select the permission which needs to be assigned to the role and create it.

Note: In production, follow the principle of least privilege for roles.

Question to Everyone : ?? why memory utilization of EC2 is not coming in cloud watch standard metrics ?

It's because AWS resources can't read details from the OS level. Eg., memory utilization are captured at OS level, so we need agent to run the EC2 Instance to send the logs to the CloudWatch.

Step 6: Assign the IAM Role.

In the EC2 dashboard, go to Actions → Security → Modify IAM Role, and assign the created role to the instance.

Step 7: Send Custom Metrics.

SSH into the instance or use EC2 Instance Connect, and execute the following command to send memory utilization metrics:

/home/ec2-user/aws-scripts-mon/mon-put-instance-data.pl --mem-util --mem-used --mem-avail

Step 8: View Custom Metrics in CloudWatch.

Once the metrics are sent, navigate to CloudWatch and view the custom metrics under a custom namespace.

The above steps will provide memory utilization metrics for the EC2 instance. These metrics can be used to create automated dashboards in CloudWatch, or you can customize your own dashboards.

Conclusion

Now I hope we understand the power of CloudWatch!

Why Should You Care? ??♀?

Because CloudWatch isn’t just about metrics and alarms—it’s about peace of mind. It keeps you informed, helps optimize resources, and ensures you never miss a critical system event. In short, it’s your personal trainer for AWS systems. ???♂?

Thank you for joining me on this AWS CloudWatch adventure! ?? Whether you’re a seasoned AWS pro or just starting your cloud journey, monitoring is the key to unlocking a resilient and optimized system. Keep experimenting, keep exploring, and most importantly, keep learning.

Until next time, happy monitoring! ??

- Jana