Amazon S3 Event Notifications

S3 Event Notifications is a feature that allows us to be notified when specific events happen in our S3 bucket, such as when an object is created, deleted, or a replication event occurs.

Some of the Event types

We can configure notifications for various event types such as:

• Object Created (s3:ObjectCreated:*)
• Object Removed (s3:ObjectRemoved:*)
• Object Restore (s3:ObjectRestore:*)
• Replication events (s3:Replication:*)
• …

Destinations of the Notification

SNS topic: Broadcast the notification to multiple subscribers.

SQS queue: Store the event messages in a queue.

Lambda function: Automatically invoke a Lambda function to process the event in real time.

Amazon EventBridge: publish the event to a serverless event bus and the event can be forwarded to over 18 AWS services as destinations. We can also have advanced filtering options with JSON rules (by inspecting the object metadata, size, and name…). EventBridge can also archive, and replay events with reliable delivery)

?

IAM Permission for each Destination

We are not using IAM roles, instead, we use resource access policy for each type of destination (these policies are similar to S3 bucket policies)

For SNS, SQS, and Lambda, we need to grant S3 the necessary permissions to call API to publish messages to an SNS topic, an SQS queue, or a Lambda function.

On the other hand, S3 does not require any additional permissions to deliver events to EventBridge.

Why use S3 Event Notifications?

Automated Processing: Automatically trigger processes when objects are uploaded or modified.

Monitoring and Auditing: receiving notifications when the objects are deleted, or modified is especially helpful for auditing and monitoring.

Decoupled Architecture: we can make use of SNS, SQS, and EventBridge to implement Event Driven Architecture, which decouples systems from the different parts of the application without direct integration.

Use cases

Image Processing: whenever an image is uploaded, S3 Event triggers a Lambda function to generate the image thumbnails.

Data Archiving: automatically moves a recently large uploaded data file to another storage class (such as Glacier or another cold storage class).

Replication Monitoring: notify us about the events that occur when objects fail to replicate between buckets.