What is SQS:
Amazon Simple Queue Service (SQS) is a managed message queuing service technical professionals and developers use to send, store and retrieve multiple massages of various sizes asynchronously.
- SQS is fast, reliable, fully managed message Queue service.
- It is a webserver that gives you access to message Queue that store messages waiting to be processed.
- It offers a reliable, highly scalable, hosted Queue for storing messages between servers.
- It allows the decoupling of application components such that a failure in one component does not cause a bigger problem to application functionality [like in coupled app] using SQS, you no longer need a highly available message cluster or the burden of running it.
- You can delete all the message in an SQS Queue without detecting the SQS Queue itself.
- You can use application on EC2 instances to read and process the SQS Queue message.
- You can use autoscaling to scale the EC2 fleet processing the SQS messages, as the Queue size increases.
- These applications on EC2 instances can process the SQS message/jobs then post the result to other SQS Queues or other AWS service.
AWS Queue Types:
AWS offers two types of queues:
1.??Standard Queues (default)
2.??FIFO Queues (First-In-First-Out)
1. Standard Queues:
- Standard Queues are the default queues offered by SQS.
- In the standard queue, we can have an unlimited number of transactions per second.
- Messages would be delivered at least once, that’s the sole job of the standard queue and it guarantees that messages would be delivered, but however sometimes we see the redundancy in messages. It doesn’t guarantee that it will deliver only one instance of a message sometimes more than one copy of a message might get delivered. The redundant message would be out of the order.
- Standard queues provide best-efforts in order that the messages would be in order. But it’s not guaranteed that it will happen sometimes a message can come out of order.
2.?FIFO Queues (First-In-First-Out):
- The FIFO Queue is like advancement over standard queues.
- The order of the messages is maintained in FIFO i.e, you will receive the message in the same order in which you have sent.
- There is no problem with the redundancy of a message. Only one message will be available to consume and it will remain in the queue till it’s consumed by the consumer.
- FIFO guarantees of ordering, uniqueness of messages (no redundancy).
- It allows grouping multiple ordered messages within a single Queue.
- FIFO has a limited number of transactions per second i.e, 300, unlike standard queues where you can have any number of transactions.
?SQS Pricing:
The first million monthly requests are free, after that pricing is according to regions for example:
- Standard Queue ----> $ 0.40/million request
- FIFO Queue ----> $ 0.50/million request.
?SQS Visibility Timeout:
- This is the amount of time the message will be invisible in the SQS after a component has read and performed the job.
- If processing is done before the timeout expires, the message then will be deleted from the Queue.
- If the job is not done then the message will be visible again and some other component can take up the job and the same message would be introduced twice.
- The default visibility is 30 seconds.
- Visibility timeout can be increased up to 12 hours.
?SQS – Retention Period:
SQS messages can remain in the Queue for upto 14 days [SQS retention Period].
- Range is 1 min to 14 days [Default is 4 days].
- Once the maximum retention period of message is reached, it will be deleted automatically from the Queue.
- Messages can be sent to the Queue and read from the Queue simultaneously.
- SQS can be used with DynamoDB, EC2, ECS, Redshift, RDS, Lambda, S3 to make distributed/decoupled applications.
- You can have multiple Queues with different priorities.
What is poll-based messaging:
We want to assume that poll-based messaging is a minor case. We might all be aware of that. For example, you would like to mail a letter with a message to a family member, so after writing the message on the letter, we put a stamp on it and hand it off to the post office.??Later, the post office takes our letter and delivery it to the final destination mailbox. Then, whenever your family member feels prepared, they can go to the mailbox, pick up that letter, and read it. That is actually what poll-based messaging means.?
In this process, a producer writes the messages, for example, a web page (frontend), and takes a message in, writes said the letter message into an existing SQS queue, later then the backend server (the consumer) can reach retrieve that message from the SQS queue whenever it’s ready. So, we can consider SQS sort of like a postman putting a letter into the mailbox.
1. Short Polling:
- A request is returned immediately even if the Queue is empty.
- It dose not wait for messages to appear in the Queue.
- It queries only a subnet of the available servers for messages [based on weighted random distribution].
- Receive messages wait time is set to 0.
- More request are used, which implies higher cost.
2. Long Polling:
- Is preferred to regular/short polling. It uses fewer requests and request cost by –
- Elemenating false empty responses by Queuing all the servers.
- Reduce the number of empty response, by allowing amazon SQS to wait until a messages is available in the Queue before sending a response unless the connection timeout (20 sec).
- Receive message wait time is set to a non-zero value [max 20 seconds]
- Billing is same for both pollings.
?Delivery Delay:
AWS SQS provides delivery delay options to postpone the delivery of new messages to a Queue. If delivery delay is defined for a Queue, any new messages will not be visible to the server for the duration of delay. The default (min.) delay for a Queue is 0 second. The maximum is 15 minutes.
Receive message-wait Time:
The default time is 0 seconds. This is maximum amount of time that a long polling receive call will wait for a message to become available before returning an empty response [max value is 20 sec].
Dead Letter Queue:?
The main task of a dead letter Queue is handling message failure. A dead letter Queue lets you set a side and isolate messages that can’t be processed correctly to determine why their processing didn’t success.
- Don’t use a dead letter Queue with a FIFO Queue, if you don’t want to break the exact order of messages or operations.
- DLQ must be of the same type as the source Queue [standard or FIFO].
How Amazon SQS Charges:
- API Action: Every Amazon SQS action count as a request.
- FIFO Request: API actions for sending, receiving, deleting and changing visibility of messages from FIFO Queue are charged at FIFO rates.
- Content of Request: A single request can have from 1 to 10 messages, upto a maximum total payload of 256 KB.
- Size of payload: Each 64 KB Chunk of a payload is billed as 1 request. For eg – API action with a 256 KB payload is billed as 4 request.
- Interaction with Amazon S3.
- Interaction with AWS KMS.
?Important Points to remember:
- SQS is not push-based, it is basically on pull-based.
- The size of messages in SQS are 256 KB.
- There is default retention period in SQS. The default retention period is of 4 days.
- Messages is SQS of a Queue is kept from 1 minute to 14 days.
- SQS also guarantees that the messages will not be processed at least once.
Thanks for reading! I hope you find this article helpful.
