Amazon SQS Overview

Introduction to Amazon SQS

Amazon Simple Queue Service (SQS) is a managed queuing service that provides a buffer layer for decoupling producers and consumers. SQS operates with a simple structure: a queue that holds messages. These messages are sent by producers and retrieved by consumers.

Key Components

Producers: Are applications that send messages to the SQS queue using the SendMessage API. Multiple producers can send multiple types of messages.

Consumers: Consumers are applications that retrieve messages from the queue by polling. They use the ReceiveMessage API to fetch messages, process them, and then delete them from the queue in order to complete the message lifecycle. Consumers can be an application hosted on EC2, on-premises servers, or Lambda functions.

Features of SQS

Unlimited Throughput: SQS supports unlimited throughput, allowing for an infinite number of messages to be stored and processed in parallel.

Message Retention: Messages are retained for a default of 4 days, with a maximum of 14 days. After this, unprocessed messages are automatically removed from the queue.

Low Latency: < 10 ms on publish and receive, ensuring quick message delivery and processing.

At-Least-Once Delivery: SQS uses an "at-least-once" delivery semantic, meaning messages may occasionally be delivered more than once, requiring consumers to handle potential duplicates.

Best-Effort Ordering: SQS will strive for best-effort ordering. For strict ordering, other queue types may be preferable.

Message Size: Maximum 256KB/message.

Scaling and Integration

Horizontal Scaling: SQS enables horizontal scaling by allowing multiple consumers to retrieve and process messages in parallel. By leveraging an Auto Scaling group (ASG), the number of consumers can dynamically increase based on queue length (ApproximateNumberOfMessages metric in CloudWatch).

Decoupling for Scalability: Decoupling applications using SQS is a great way to prevent bottlenecks and improve performance.

Security

Encryption: Encryption in transit through HTTPS and at rest with KMS. For client-side encryption, the client application must handle encryption and decryption itself.

Access Control: Access to SQS is managed via IAM policies, and cross-account access can be controlled using SQS access policies, similar to S3 bucket policies. This enables other AWS services, like SNS or S3, to interact with SQS for tasks like sending event notifications.