AWS Lambda Project: Download CSV Report of all instances with 1-Click

This project was assigned to me by my team lead Naufal Ali Shah and it helped me to further polish my AWS lambda skills.

Scenario:

There are 100 or more EC2 instances in our AWS account. We need a document (preferably in CSV format) having each instance’s name, instance ID, key pair the instance is using and public IP address.?

Solution:

I need to develop a lambda function that should read all the required properties (instance name, instance ID, key_pair and public ipv4 address) from created EC2 instances and provide all this info in a downloadable CSV format.

Step 1: Create a Lambda Function:

I prefer python to work with, so while creating lambda function, I chose the runtime as ‘Python 3.12’.?

I am not getting into the details of each step in creating a lambda function so I named the lambda function ‘GetEC2InstanceDetails’ , left other options as default and created the function.?

Step 2: Allow lambda to Access EC2:

When we create a new function, it is created by default with ‘AWSLambdaBasicExecutionPermission’ and automatically adds a role in IAM.?

But in order to read EC2 details, we need to give our lambda function an additional permission which is ‘AmazonEC2ReadOnlyAccess’.?

Go to IAM, We can see AWS created a role with function name as prefix, open it and add the necessary permission.?

Step 3: Lets Code:

Once we have the necessary permissions, let’s go back to lambda, open the created function and start writing some code.

Remove the code that’s already there and let’s start by importing the necessary libraries.?

import json
import boto3        

'boto3' is the AWS SDK for python, which means, we can use this library to interact with AWS services. 'json' library is imported because lambda expects the final response/output to be json formatted.

Let's now write a function named get_ec2_instances(). This function will return all the required information (name, instance id, public ipv4 address and associated key_pair) and save it in a list named instances.

def get_ec2_instances():
    ec2 = boto3.client('ec2')
    response = ec2.describe_instances()

    instances = []
    for reservation in response['Reservations']:
        for instance in reservation['Instances']:
            name = 'N/A'
            for tag in instance.get('Tags', []):
                if tag['Key'] == 'Name':
                    name = tag['Value']

            key_pair_name = instance.get('KeyName', 'N/A')

            instances.append({
                'name': name,
                'id': instance['InstanceId'],
                'publicIp': instance.get('PublicIpAddress', 'N/A'),
                'keyPair': key_pair_name,
            })

    return instances
        

if you look at the code, you will notice that we are using describe_instances() method from boto3 library. This will NOT work, if you haven't added the necessary permissions described in Step 2 above.

if you run the code at this point, it will return an error. Why?

Because each lambda function expects a lambda_handler() function which is the entrypoint for the code. That means, when executed, lambda function will basically look for lambda_handler() and execute the code written inside.

We can even change this behavior by editing the 'Runtime settings' but at this point, this is not required.

Before writing the lambda_handler() function, let's pause here and add 'API Gateway' as a trigger to our function. This will provide us a unique URL, which we can use to call our lambda function.

Step 4: Adding API Gateway as Trigger:

Adding an API Gateway as trigger is easy. Just click on "Add trigger", choose 'API Gateway' -> Create a new HTTP API, choose security as 'Open' and Done.

In the image, you can see the 'API Endpoint'. We can use this API endpoint to trigger our lambda function, when necessary. Let's now go back and complete the code.

Step 5: Back to Code:

Let's write the lambda_handler() function.

def lambda_handler(event, context):
    instances = get_ec2_instances()

    html_response = """
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <title>EC2 Instances</title>
        <style>
            table {
                border-collapse: collapse;
                width: 100%;
            }

            th, td {
                border: 1px solid #dddddd;
                text-align: left;
                padding: 8px;
            }

            th {
                background-color: #f2f2f2;
            }
        </style>
    </head>
    <body>
        <div class="header-row">
            <h2>EC2 Instances</h2>
            <button onclick="downloadCSV()">Download CSV</button>
        </div>
        <table id="instancesTable">
            <tr>
                <th>Name</th>
                <th>ID</th>
                <th>Public IP</th>
                <th>Key Pair</th>
            </tr>
    """

    for instance in instances:
        html_response += f"""
            <tr>
                <td>{instance['name']}</td>
                <td>{instance['id']}</td>
                <td>{instance['publicIp']}</td>
                <td>{instance['keyPair']}</td>
            </tr>
        """
return {
        'statusCode': 200,
        'headers': {'Content-Type': 'text/html'},
        'body': html_response
    }
        

This will first call the get_ec2_instances() function we have written before and save all the details in instances.

From the code, you can see that I am using HTML to populate the page with relevant information in tabular format. I have also added some style to the HTML - to make it look good (which I copied from online :p ).

Also, inside the <body> , you can see that I am using a 'Download CSV ' button, but this button will currently not do anything, since we haven't added any logic for this button.

Now, lets use some Javascript to add the functionality behind 'Download CSV ' button. Add the following script before return statement in lambda_handler() function.

 html_response += """
        </table>

        <script>
            function downloadCSV() {
                var csvContent = "data:text/csv;charset=utf-8,";
                csvContent += "Name,ID,Public IP,Key Pair\\n";

                var instances = """ + json.dumps(instances) + """;

                instances.forEach(function(instance) {
                    csvContent += instance.name + "," + instance.id + "," + instance.publicIp + "," + instance.keyPair + "\\n";
                });

                var encodedUri = encodeURI(csvContent);
                var link = document.createElement("a");
                link.setAttribute("href", encodedUri);
                link.setAttribute("download", "ec2_instances.csv");
                document.body.appendChild(link);
                link.click();
            }
        </script>
    </body>
    </html>
    """        

Step 6: Test:

We can add the following code below lambda_handler() to see the response of our code before triggering it via API endpoint.

if __name__ == '__main__':
    print(lambda_handler(None, None)['body'])        

This is actually calling our lambda_handler() and we are printing the <body> from the response. Click on blue "Test" button to test the code. We will see something like this:

Test Successful! We can see the HTML response in output. Let's now use the API endpoint we have created in Step 4 to see something beautiful.

Click on the 'Download CSV ' button and get this nice table downloaded in the form of CSV.

Conclusion:

Whole code is also available on my Github profile:

AWS Lambda Project

We can use the Downloaded CSV for some further automation. If you have any ideas in mind, don't hesitate to share. Liked the Project? Give a thumbs up.