Optimizing AWS Costs: Automated EBS Snapshot Deletion Using Lambda Functions

Abstract

Effective management of AWS resources is crucial for maintaining cost efficiency and operational integrity. Unused Elastic Block Store (EBS) snapshots can accumulate, leading to increased costs and cluttered environments. This paper introduces an automated solution utilizing AWS Lambda functions to clean up stale EBS snapshots. By implementing this automated process, organizations can significantly reduce storage costs, enhance system efficiency, and allocate more resources to core business tasks.

Step 1: Policy Creation and Role Assignment

To allow the Lambda function to describe and delete EBS snapshots and volumes, you must create an AWS Identity and Access Management (IAM) policy. Attach this policy to the default role created by the Lambda function.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeSnapshots",
                "ec2:DeleteSnapshot",
                "ec2:DescribeVolumes",
                "ec2:DescribeInstances"
            ],
            "Resource": "*"
        }
    ]
}        

Step 2: Lambda Function Code

The Lambda function iterates through all EBS snapshots, checks their attachment status, and deletes snapshots that are not associated with any volume or attached to volumes not linked to running EC2 instances.

import boto3

def lambda_handler(event, context):
    ec2 = boto3.client('ec2')

    # Get all EBS snapshots
    response = ec2.describe_snapshots(OwnerIds=['self'])

    # Get all active EC2 instance IDs
    instances_response = ec2.describe_instances(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])
    active_instance_ids = set()

    for reservation in instances_response['Reservations']:
        for instance in reservation['Instances']:
            active_instance_ids.add(instance['InstanceId'])

    # Iterate through each snapshot and delete if it's not attached to any volume or the volume is not attached to a running instance
    for snapshot in response['Snapshots']:
        snapshot_id = snapshot['SnapshotId']
        volume_id = snapshot.get('VolumeId')

        if not volume_id:
            # Delete the snapshot if it's not attached to any volume
            ec2.delete_snapshot(SnapshotId=snapshot_id)
            print(f"Deleted EBS snapshot {snapshot_id} as it was not attached to any volume.")
        else:
            # Check if the volume still exists
            try:
                volume_response = ec2.describe_volumes(VolumeIds=[volume_id])
                if not volume_response['Volumes'][0]['Attachments']:
                    ec2.delete_snapshot(SnapshotId=snapshot_id)
                    print(f"Deleted EBS snapshot {snapshot_id} as it was taken from a volume not attached to any running instance.")
            except ec2.exceptions.ClientError as e:
                if e.response['Error']['Code'] == 'InvalidVolume.NotFound':
                    # The volume associated with the snapshot is not found (it might have been deleted)
                    ec2.delete_snapshot(SnapshotId=snapshot_id)
                    print(f"Deleted EBS snapshot {snapshot_id} as its associated volume was not found.")        

Going through each line of code

import boto3        

Explanation: This line imports the boto3 library, which is the Amazon Web Services (AWS) SDK for Python. This library allows Python scripts to interact with various AWS services programmatically.

def lambda_handler(event, context):        

Explanation: This defines a function named lambda_handler that takes two parameters, event and context. This function is the entry point for AWS Lambda functions. When the Lambda function is triggered, AWS passes event data to this function.

    ec2 = boto3.client('ec2')        

Explanation: This line creates a client object for the Amazon EC2 service using boto3.client('ec2'). The ec2 client object is used to interact with the EC2 service

    response = ec2.describe_snapshots(OwnerIds=['self'])        

Explanation: This line calls the describe_snapshots method on the ec2 client to retrieve a list of all EBS (Elastic Block Store) snapshots owned by the account. The parameter OwnerIds=['self'] specifies that only snapshots owned by the current AWS account should be returned.

    instances_response = ec2.describe_instances(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])        

Explanation: This line calls the describe_instances method on the ec2 client to get information about all EC2 instances that are currently running. The Filters parameter is used to specify that only instances with the state 'running' should be returned.

    active_instance_ids = set()        

Explanation: This initializes an empty set named active_instance_ids. This set will be used to store the IDs of all running EC2 instances.

    for reservation in instances_response['Reservations']:
        for instance in reservation['Instances']:
            active_instance_ids.add(instance['InstanceId'])        

Explanation: This nested loop iterates through the instances returned in instances_response. The response contains a list of reservations, and each reservation contains a list of instances. For each instance, its ID (InstanceId) is added to the active_instance_ids set.

    for snapshot in response['Snapshots']:
        snapshot_id = snapshot['SnapshotId']
        volume_id = snapshot.get('VolumeId')        

Explanation: This loop iterates through each snapshot in the list of snapshots obtained from the describe_snapshots call. For each snapshot, the SnapshotId and VolumeId are retrieved. snapshot.get('VolumeId') returns the volume ID if it exists, or None if it doesn't.

        if not volume_id:
            ec2.delete_snapshot(SnapshotId=snapshot_id)
            print(f"Deleted EBS snapshot {snapshot_id} as it was not attached to any volume.")        

Explanation: This checks if the volume_id is None (i.e., the snapshot is not attached to any volume). If so, it calls the delete_snapshot method to delete the snapshot and prints a message indicating the deletion.

        else:
            try:
                volume_response = ec2.describe_volumes(VolumeIds=[volume_id])
                if not volume_response['Volumes'][0]['Attachments']:
                    ec2.delete_snapshot(SnapshotId=snapshot_id)
                    print(f"Deleted EBS snapshot {snapshot_id} as it was taken from a volume not attached to any running instance.")        

Explanation: If the snapshot is attached to a volume (volume_id is not None), it attempts to describe the volume using describe_volumes. If the volume exists but has no attachments (i.e., it's not attached to any instance), it deletes the snapshot and prints a message.

            except ec2.exceptions.ClientError as e:
                if e.response['Error']['Code'] == 'InvalidVolume.NotFound':
                    ec2.delete_snapshot(SnapshotId=snapshot_id)
                    print(f"Deleted EBS snapshot {snapshot_id} as its associated volume was not found.")        

Explanation: This try-except block handles exceptions that might occur when describing the volume. If the volume is not found (indicated by the InvalidVolume.NotFound error code), it deletes the snapshot and prints a message.

Results

Implementing this Lambda function results in several key benefits:

- Cost Optimization: Significant reduction in storage costs by removing unused snapshots.

- Enhanced Operational Efficiency: Automated snapshot management ensures a clutter-free and organized AWS environment.

- Resource Reallocation: Freed-up resources can be redirected to core business activities, enhancing overall productivity.

Conclusion

The automation of EBS snapshot cleanup using AWS Lambda functions offers a robust solution to manage AWS storage resources efficiently. By leveraging this approach, organizations can achieve substantial cost savings and improve the operational efficiency of their cloud environments. Future work may explore the integration of this function with other AWS services to further enhance automation and resource management capabilities.

References

- Amazon Web Services, Inc. (n.d.). AWS Lambda Documentation. Retrieved from https://docs.aws.amazon.com/lambda/

- Amazon Web Services, Inc. (n.d.). Amazon Elastic Block Store (EBS) Documentation. Retrieved from https://docs.aws.amazon.com/ebs/