Unlock Cost Efficiency in the Cloud: A FinOps Revelation for EBS Optimization!

In today's rapidly evolving cloud landscape, navigating the complexities of cloud costs has become a critical challenge for many organizations. We've taken a groundbreaking step forward with our latest initiative aimed at revolutionizing how we manage and optimize our cloud infrastructure, particularly focusing on the often overlooked aspect of Elastic Block Store (EBS) volumes.

Our recent endeavors have unveiled a significant opportunity for cost optimization by identifying and managing unattached AWS EBS volumes. Through a sophisticated blend of automation and strategic insight, we developed a two-pronged approach that not only highlights unutilized resources but also paves the way for their efficient management.

Here’s a step-by-step overview of our process:

1. List All Unattached EBS Volumes: Initiating our cost-saving journey, we use the AWS SDK to pinpoint all EBS volumes languishing in an unattached state, readying them for review.
2. Gather Volume Details: We meticulously compile details such as volume ID, size, creation time, and associated tags for each unattached volume, laying the groundwork for informed decision-making.
3. Generate an Excel Report: Leveraging these details, we craft a comprehensive Excel report, presenting a clear and actionable snapshot of potential cost savings to our management team.
4. Prepare Email Content: An informative email, detailing the purpose and findings of our report, is prepared for dispatch to the relevant stakeholders.
5. Attach Excel Report to Email: The report is then attached to the email, ensuring our findings are communicated effectively and efficiently.
6. Send Email: Using AWS SES, we dispatch the email, complete with its critical attachment, heralding the first step towards significant cost optimization.
7. Receive Delete Request: Following a thorough review, specific volumes are earmarked for deletion, a crucial stride towards operational efficiency.
8. Attempt to Delete Volumes: Our script then swings into action, attempting to delete each selected volume, with every success and hiccup carefully logged.
9. Return Deletion Results: A summary of the deletion attempts is compiled, offering a transparent view of the action taken and its outcomes.
10. Communicate Outcome: This summary is communicated back, confirming the efficacy of our approach and the successful reclamation of wasted resources.

By embedding FinOps principles into our operations, we are setting new benchmarks for cost optimization, operational excellence, and environmental stewardship in the cloud. Our journey doesn't end here. This initiative is a testament to our commitment to continuous improvement and innovation in cloud financial management.

#CloudFinOps #CostOptimization #AWS #Innovation

Demo-

Step 1-> Create a Lambda function to automatically retrieve details of unattached EBS volumes and send them via email to the respective team members(Give appropriate permission to lambda execution role like "ec2:DescribeVolumes", "ses:SendRawEmail")

import boto3

from openpyxl import Workbook

import io

from email.mime.multipart import MIMEMultipart

from email.mime.text import MIMEText

from email.mime.application import MIMEApplication

def list_unattached_volumes():

ec2 = boto3.client('ec2')

volumes = ec2.describe_volumes(Filters=[{'Name': 'status', 'Values': ['available']}])

volume_details = []

for volume in volumes['Volumes']:

details = {

'VolumeId': volume['VolumeId'],

'Size': volume['Size'],

'CreateTime': volume['CreateTime'].strftime("%Y-%m-%d %H:%M:%S"),

'Tags': volume.get('Tags', [])

}

volume_details.append(details)

return volume_details

def generate_excel(volume_details):

output = io.BytesIO()

wb = Workbook()

ws = wb.active

ws.title = "Unattached EBS Volumes"

headers = ['Volume ID', 'Size (GiB)', 'Create Time', 'Tags']

ws.append(headers)

for volume in volume_details:

tags = ', '.join([f"{tag['Key']}: {tag['Value']}" for tag in volume['Tags']])

ws.append([volume['VolumeId'], volume['Size'], volume['CreateTime'], tags])

wb.save(output)

output.seek(0)

return output

def send_email_with_attachment(ses_region, sender, recipient, subject, body_text, attachment, file_name):

client = boto3.client('ses', region_name=ses_region)

msg = MIMEMultipart()

msg['Subject'] = subject

msg['From'] = sender

msg['To'] = recipient

# Attachment part

part = MIMEApplication(attachment.getvalue())

part.add_header('Content-Disposition', 'attachment', filename=file_name)

msg.attach(part)

# Body part

part = MIMEText(body_text, 'plain')

msg.attach(part)

response = client.send_raw_email(

Source=sender,

Destinations=[recipient],

RawMessage={'Data': msg.as_string()}

)

return response

def lambda_handler(event, context):

volume_details = list_unattached_volumes()

if not volume_details:

print("No unattached volumes found.")

return

excel_file = generate_excel(volume_details)

ses_region = 'us-east-1'

sender = '[email protected]'

recipient = '[email protected]'

subject = 'Unattached EBS Volumes Report'

body_text = "Please find the attached report on unattached EBS volumes."

file_name = 'Unattached_EBS_Volumes.xlsx'

send_email_with_attachment(ses_region, sender, recipient, subject, body_text, excel_file, file_name)

print("Email sent with attachment successfully!")

Step-2 Add Lambda layer "pandas" module for 'openpyxl'

Step 3- Configure your SES and provide your ses_region, sender and receiver in the lambda function.

Step 4 - Create an Event-bridge rule to trigger the lambda function(You can set it every 7 days as per requirement)

Step 5 - Now, the respective team members will receive an email with the details of unattached EBS volumes in an Excel sheet

Step 6 - After the teams have reviewed and verified the volumes according to the approval process, unwanted EBS volumes can now be deleted by providing their volume ID to another Lambda function. Ensure that appropriate execution permissions are granted.(Give appropriate Execution permission.)

import boto3

def delete_ebs_volumes(volume_ids):

"""

Deletes EBS volumes based on a list of volume IDs.

Parameters:

- volume_ids (list): A list of EBS volume IDs to delete.

"""

ec2 = boto3.client('ec2')

deleted_volumes = []

for volume_id in volume_ids:

try:

ec2.delete_volume(VolumeId=volume_id)

print(f"Successfully deleted volume: {volume_id}")

deleted_volumes.append(volume_id)

except Exception as e:

print(f"Failed to delete volume {volume_id}: {e}")

return deleted_volumes

def lambda_handler(event, context):

"""

AWS Lambda event handler.

Expects an event object with a key 'VolumeIds', a list of EBS volume IDs to delete.

"""

if 'VolumeIds' not in event:

raise ValueError("Event does not contain 'VolumeIds'.")

volume_ids = event['VolumeIds']

if not volume_ids:

raise ValueError("VolumeIds list is empty.")

deleted_volumes = delete_ebs_volumes(volume_ids)

return {

'statusCode': 200,

'body': {

'message': 'Delete operation completed.',

'deletedVolumes': deleted_volumes

}

}

Step 7 - Name the event 'test' and provide the IDs of the EBS volumes you wish to delete

Step 8 - Run the Lambda it will delete all Unwanted EBS.