Securing Amazon Simple Storage Service (Amazon S3) - Part 3

This is the third and final part of our series on securing Amazon S3. In the last two articles, we discussed three topics: what Amazon S3 is, the Amazon S3 security best practices are, and configured the AWS Terraform Provider and variables necessary to build the AWS resources. We also reviewed the Terraform code used to deploy the AWS resources and successfully deployed the Terraform Code to AWS.

In this final article, we will validate that the Amazon S3 best practices were followed when deploying the code with Terraform, review the findings for Amazon Macie and AWS Config findings, and validate that email notifications are being sent for Amazon Macie and AWS Config findings. Last, we will run some tests to see if we can detect any changes to the Amazon S3 buckets.

You can access all of the code used in my GitHub repository.

Subscribe to the Amazon SNS Topic

Before validating the deployment, we must ensure notifications can be sent using Amazon SNS.

To receive notifications from Amazon Macie and AWS Config, you will need to subscribe to the Amazon SNS Topic. You will receive a similar email notification confirming your Amazon SNS Topic subscription.

Your browser should open to display, showing the subscription confirmation is successful. This is an example of what it should look like.

Validate Amazon S3 bucket settings.

Validating the settings of an Amazon S3 bucket can be confirmed in a few ways, such as the AWS CLI and AWS Console in a web browser.?

To confirm the settings of the Amazon S3 "dallin-s3-lab" bucket through the AWS Console, follow these steps.

Step 1. Log into AWS Console and switch to the AWS region used during the Terraform code deployment

Step 2. Switch to Amazon S3 in the Console and locate Amazon S3 buckets.

Step 3. Click on "dallin-s3-lab" S3 bucket.

Step 4. Click on the "Objects" tab to check if sensitive files were uploaded

Step 5. Click on the "Properties" tab to check for the following

• Bucket Versioning
• Default Encryption
• Server access logging
• AWS CloudTrail data events

Step 6. Click on the "Permissions" tab to check for the following

• Permissions overview
• Block public access (bucket settings)
• Bucket policy
• Object Ownership
• Access control list (ACL)

Step 7. Click on the "Management" tab to check for the following

• Lifecycle rules

To confirm the settings of the Amazon S3 "dallin-s3-lab" bucket through AWS CLI, follow these steps.?

The AWS CLI needs to be installed and configured. If it is not, you can be AWS CLI documentation here.

The AWS CLI "s3api" commands can be found here.

Step 1. Configure AWS Profile to use with AWS CLI

export AWS_PROFILE=bsisandbox        

Step 2. Example to check bucket-logging is configured

aws s3api get-bucket-logging --bucket dallin-s3-lab --no-cli-pager        

Results of the "aws s3api get-bucket-logging" command

{? 
  "LoggingEnabled": {
??? "TargetBucket": "dallin-s3-lab-logs",
??? "TargetPrefix": "logs/"
? }
}        

Step 3. The following are other "s3api" commands to use.

aws s3api get-bucket-acl --bucket dallin-s3-lab --no-cli-pager        

As you can see, using the AWS Console or AWS CLI is time-consuming, especially if you have to validate multiple Amazon S3 buckets.

A better way to validate and confirm Amazon S3 bucket configurations are correctly configured to Amazon S3 best practices is to use AWS Config.

Validate with AWS Config

Step 1. Log into AWS Console and switch to the AWS region used during the Terraform code deployment

Step 2. Switch to AWS Config

Step 3. Click on Rules and review the rules which are compliant and non-compliant resources

Step 4. Click on AWS Config rule "dallin-s3-lab-s3-event-notifications-enabled-config-rule" to check which Amazon S3 buckets are non-compliant.

Review Amazon Macie findings

Amazon Macie is an AWS resource to discover sensitive data in any Amazon S3. It helps protect your sensitive data.?

To check the Amazon Macie findings, follow these steps.

Step 1. Log into AWS Console and switch to the AWS region used during the Terraform code deployment

Step 2. Switch to Amazon Macie

Step 3. Click on Jobs to check if the classification jobs were created successfully

Step 4. Locate "Findings" on the left side and click "By bucket" to review Amazon Macie's findings.

Step 5. Click on "dallin-s3-lab" bucket, and you will see that it found 2 High severity alerts.

Step 6. Click on Findings by severity "High" on the right side of the web browser

Step 7. After clicking on the High severity link, you will see the results of what S3 object is affected.?

Amazon Macie is another powerful tool AWS provides to help protect your data stored in Amazon S3 buckets. You can create job classifications using AWS Managed data identifiers and custom data identifiers you make, and the jobs can be scheduled or run once. All of the results from the findings for Amazon Macie can be exported to an AWS S3 bucket you specify.?

Notifications from AWS Config and Amazon Macie

You need to be notified to take the necessary actions quickly on any AWS Config or Amazon Macie finding. AWS provides several options, but Amazon Eventbridge is the easiest to set up and configure. As we confirmed our SNS Topic subscription, we should have received several email notifications on the AWS Config and Amazon Macie findings. This is possible due to the configuration we did in our previous article to set up Amazon Eventbridge and Amazon SNS.

As you can see, I received several SNS notifications for the findings from AWS Config and Amazon Macie.

By subscribing to an Amazon SNS topic, you will receive email notifications on any findings from AWS Config and Amazon Macie.?

Test Amazon S3 Bucket changes

Let's manually change a setting in the Amazon S3 "dallin-s3-lab" bucket and see if AWS Config captures the change. We will delete the bucket policy in the Amazon S3 "dallin-s3-lab" bucket. This bucket policy denies accessing all HTTP requests to Amazon S3 "dallin-s3-lab" objects.

We will use the AWS CLI to complete this task.? Follow these steps to remove the bucket policy from the Amazon S3 "dallin-s3-lab" bucket.

Step 1. Configure AWS Profile to use with AWS CLI

export AWS_PROFILE=bsisandbox        

Step 2. Check bucket-policy is configured to deny HTTP requests

aws s3api get-bucket-policy --bucket dallin-s3-lab --no-cli-pager        

Results of the above command

{
??? "Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"denyInsecureTransport\",\"Effect\":\"Deny\",\"Principal\":\"*\",\"Action\":\"s3:*\",\"Resource\":[\"arn:aws:s3:::dallin-s3-lab/*\",\"arn:aws:s3:::dallin-s3-lab\"],\"Condition\":{\"Bool\":{\"aws:SecureTransport\":\"false\"}}}]}"
}        

Step 3. Remove bucket-policy

aws s3api delete-bucket-policy --bucket dallin-s3-lab --no-cli-pager        

Step 4. Check bucket-policy is deleted

aws s3api get-bucket-policy --bucket dallin-s3-lab --no-cli-pager        

Results of the above command.

An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicy operation: The bucket policy does not exist        

The bucket policy is now deleted from the Amazon S3 "dallin-s3-lab" bucket. We should receive an email notification about the configuration change from AWS Config when it re-evaluates the AWS Config rules.

We can force AWS Config to re-evaluate the rule or wait for AWS Config to alert on the rule "dallin-s3-lab-s3-bucket-ssl-requests-only-config-rule". AWS Config should alert on this fairly quickly.

To review if AWS Config detected the change, follow the following steps.

Step 1. Log into AWS Console and switch to the AWS region used during the Terraform code deployment

Step 2. Switch to AWS Config

Step 3. Click on Rules and review the rules which are compliant and non-compliant resources

Step 4. Click on AWS Config rule "dallin-s3-lab-s3-event-notifications-enabled-config-rule" to see which Amazon S3 buckets are non-compliant.

Step 5. The AWS Config rule "dallin-s3-lab-s3-event-notifications-enabled-config-rule" shows the Amazon S3 bucket "dallin-s3-lab" is non-compliant.?

Step 6. Check your email to see if an SNS Notification was received for the AWS Config violation.

AWS Config detected that the deleted bucket policy contained the policy to deny all HTTP requests to the bucket. We also received an SNS Notification of the change.

Clean Up AWS Resources

The last thing we must do is clean up the AWS resources so we are not billed for resources we no longer use. We will use Terraform to destroy the AWS resources and infrastructure to accomplish this.

Follow these steps to destroy the AWS resources and infrastructure.

NOTE: There is a known bug in the AWS Provider that Terraform cannot delete any Amazon Macie Classification Jobs scheduled to run One-Time.

Step 1. Remove terraform state for Amazon Macie Classification One-Time Job.

terraform state rm aws_macie2_classification_job.macie_one_time        

Step 2. Clean up Terraform

terraform destroy        

Conclusion

This three-part article demonstrated how to secure Amazon S3 buckets using Amazon S3 best practices. To secure it, we configured Amazon S3 bucket settings such as bucket policies, S3 access logging, and bucket versioning.?

We leveraged Amazon Macie to detect sensitive data such as credit cards or social security numbers and AWS Config to monitor the configuration standards we defined to secure our Amazon S3 buckets. Amazon SNS and Amazon Eventbridge were used to detect Amazon Macie findings and changes from AWS Config rules. They sent email notifications to users who subscribed to the Amazon SNS topic. After creating the AWS resources and infrastructure, we validated the configuration and settings through the AWS Console and AWS CLI.?

We tested if someone were to manually change a setting on an Amazon S3 bucket by removing the bucket policy containing a policy to deny all HTTP requests. AWS Config automatically detected the change, and we were emailed about the changes.

By leveraging Amazon S3 best practices, we can feel more confident that our Amazon S3 buckets are secure.