Five Ways to Reduce Your AWS Bill

Hosting your infrastructure in the cloud has its benefits in terms of auto-scaling, HA, pay-as-you-go billing etc. but if we're not careful with the monitoring of the usage, we can end up over-allocating and under-utilizing resources which can take a bite out of our budget.

The above graph is from our actual AWS cost explorer. We at Loop were able to reduce our cloud expenses by 1/3rd. Here's what we did.

1. Removed un-needed Synthetic Canaries

We have a few public facing URLs that were being monitored by Synthetic Canaries in Cloudwatch. Canaries are fancy name for pings to a URL that collect http responses and feeds to Cloudwatch. There is a charge per run. Since our QA environment is a (scaled down) replica of our prod, we setup the same canaries for QA URLs. Turns out these were adding considerably to our cost without adding much value to us (if our QA env was down, it wasn't a customer outage, and we could live with that).

We did the following to optimize this cost:

• Removed all QA canaries
• Increased the duration between runs (from every minute to every 5 minutes etc.)

2. EC2 optimization

The second contributor to our cost was EC2. On monitoring the usage, we found that we had over-allocated these boxes and the burst usage hardly came to 40% of CPU/memory.

We did the following to optimize this cost:

• Downsized the instance type to match the usage pattern accounting for the burst usage
• Setup a schedule where non-prod instance would get shutdown automatically at day end and get turned back on the next day
• Used spot instance for build servers

3. EBS Volume shrinking

The third contributor to our cost was over allocating volumes in EC2s. On monitoring the disk space on all EC2s, we found our space to be in the range of 30-50% filled. Needless to say this had to be optimized.

We did the following to optimize this cost:

• Shrink the volume accordingly on each EC2. The catch here is that EC2 volumes can be increased dynamically but cannot be reduced so we had to follow the process of allocating new volume, copying data over, unmounting the old volume and then deleting it

4. S3 compression

The fourth contributor to our cost was the data in our S3 buckets. We used the Storage Lens dashboard to isolate the buckets that had the highest volume of data. We had the choice of setting up lifecycle policies so that data older than 2 months old would be auto archived to glacier however this would have a cost per object and since we had a lot of objects, this would not be feasible for us.

We did the following to optimize this cost:

• Setup a nightly cron job that would use a nifty python utility to tar all the objects in a bucket older than n days

5. RDS optimization

The 5th contributor to our cost was the beefy RDS instance that we had provisioned that was also under-utilized. On monitoring the usage, we found that we had over-allocated this instance and the burst usage hardly came to 30-35% of CPU.

We did the following to optimize this cost:

• Downsized to an instance that would support this traffic without getting any degradation in query latency
• Did some SQL optimization after identifying the query from RDS dashboard that had the highest latency

Conclusion

I hope you found these tips helpful. What are some of the ways you have optimized your cloud costs?

#cloud #aws #startup #costsavings