AWS SQS and it's use cases

"RedBus founded in 2006 is the best platform in the world for booking bus tickets online. Currently, it is serving 6 countries around the world (India, Malaysia, Singapore, Indonesia, Peru, and Colombia)." In this article, we'll have an overview about AWS SQS services and how it's helping other companies, government organization to overcome the obstrucles.

What is AWS SQS?

AWS SQS (Amazon Simple Queue Service) is a service which helps to align the message. Moreover, it also helps to enable the user to separate and scale microservices, distributed system, and serverless applications. Amazon SQS makes it easy to manage the operating message-oriented middleware and enhances the developers to focus on their work. Amazon Simple Queue Service works at any volume without losing the message or requiring other services to be available. It helps to send, store, and receive messages between software components. AWS SQS can start with the help of the tools such as Amazon Console, command line interface, and SDK.

In Amazon SQS there are two types of queues which are Standard Queue and AWS SQS FIFO. Standard queue offers at least one delivery and maximum throughput. The FIFO queues guarantee that the processed message takes place only once in the first in first out basis.

What are the benefits of Using AWS SQS ?

1. Eliminate Administrative Overhead: AWS manages all in progress operations and underlying infrastructure required to produce an extremely accessible and scalable message queuing service. With AWS SQS, there’s no direct value, no ought to acquire, install, and assemble messaging package, and no long build-out and maintenance of supporting infrastructure. Amazon SQS queues are dynamically created and scale automatically, therefore, you’ll be able to build and grow applications quickly and with efficiency.

2. Reliably Deliver Messages: Use AWS SQS to transmit any volume of data, at any level of output, while not losing messages or requiring alternative services to be accessible. Amazon SQS helps you to decouple application parts in order that they run and fail severally, increasing the fault tolerance of the system. Multiple copies of each message area unit hold on redundantly across multiple accessible zones in order that they’re out there whenever required.

3. Keep Sensitive Information Secure: You can use Amazon SQS to exchange sensitive data between applications using server-side secret writing (SSE) to inscribe every message body. The AWS SQS compass point integration with AWS Key Management Service (KMS) permits you to centrally manage the keys that defend SQS messages together with keys that defend your alternative AWS resources. AWS KMS logs each use of your encryption keys to AWS CloudTrail to assist meet your restrictive and compliance wants.

4. Scale Elastically and Cost-Effectively: AWS SQS leverages the AWS cloud to dynamically scale supported demand. Amazon SQS scales elastically together with your application. Therefore, you don’t have to worry regarding capability designing and pre-provisioning. There’s no limit to the number of messages per queue, and commonplace queues offer nearly unlimited output. Prices area unit supported usage that provides important value saving versus the “always-on” model of self-managed electronic messaging middleware.

Key Features of AWS SQS services

• At-Least-Once Delivery - A message in the queue is delivered at least once. Message delivery is guaranteed, no message is lost.[Text Wrapping Break]
• Multiple components can work on a single queue. SQS uses a lock mechanism, if one component is using a message, it is made hidden to other components. Upon successful processing, message is deleted from the queue. If the message processing fails, it stays in the queue and is made visible to all the components. This feature is called Visibility Timeout.
• There are two types of queues - Standard and FIFO. In standard queue the messages are picked up randomly. It might not be in the order it entered the queue while FIFO queue uses first-in-first-out, it ensures the order.
• For the messages that cannot be processed are kept in dead-letter queue.
• Billing is done based on the number of requests to the queue. SQS is a good service to be used for applications to increase efficiency, reliability and performance. 

Functionality of AWS SQS Services

• Unlimited queues and messages: Create unlimited Amazon SQS queues with an unlimited number of message in any region
• Payload Size: Message payloads can contain up to 256KB of text in any format. Each 64KB ‘chunk’ of payload is billed as 1 request. For example, a single API call with a 256KB payload will be billed as four requests. To send messages larger than 256KB, you can use the Amazon SQS Extended Client Library for Java, which uses Amazon S3 to store the message payload. A reference to the message payload is sent using SQS.
• Batches: Send, receive, or delete messages in batches of up to 10 messages or 256KB. Batches cost the same amount as single messages, meaning SQS can be even more cost effective for customers that use batching.
• Long polling: Reduce extraneous polling to minimize cost while receiving new messages as quickly as possible. When your queue is empty, long-poll requests wait up to 20 seconds for the next message to arrive. Long poll requests cost the same amount as regular requests.
• Retain messages in queues for up to 14 days.
• Send and read messages simultaneously.
• Message locking: When a message is received, it becomes “locked” while being processed. This keeps other computers from processing the message simultaneously. If the message processing fails, the lock will expire and the message will be available again.
• Queue sharing: Securely share Amazon SQS queues anonymously or with specific AWS accounts. Queue sharing can also be restricted by IP address and time-of-day.
• Server-side encryption (SSE): Protect the contents of messages in Amazon SQS queues using keys managed in the AWS Key Management Service (AWS KMS). SSE encrypts messages as soon as Amazon SQS receives them. The messages are stored in encrypted form and Amazon SQS decrypts messages only when they are sent to an authorized consumer.
• Dead Letter Queues (DLQ): Handle messages that have not been successfully processed by a consumer with Dead Letter Queues. When the maximum receive count is exceeded for a message it will be moved to the DLQ associated with the original queue. Set up separate consumer processes for DLQs which can help analyze and understand why messages are getting stuck. DLQs must be of the same type as the source queue (standard or FIFO).

How AWS SQS Service works ?

There are three main parts in a distributed messaging system: the components of your distributed system, your queue (distributed on Amazon SQS servers), and the messages in the queue.

In the following scenario, your system has several producers (components that send messages to the queue) and consumers (components that receive messages from the queue). The queue (which holds messages A through E) redundantly stores the messages across multiple Amazon SQS servers.

A producer (component 1) sends message A to a queue, and the message is distributed across the Amazon SQS servers redundantly.

 When a consumer (component 2) is ready to process messages, it consumes messages from the queue, and message A is returned. While message A is being processed, it remains in the queue and isn't returned to subsequent receive requests for the duration of the visibility timeout.

 The consumer (component 2) deletes message A from the queue to prevent the message from being received and processed again when the visibility timeout expires.

Use cases of AWS SQS Service

Many companies and government organization has used AWS SQS service and they all were benefitted by one or another ways. Let's have a look at them,

> RedBus:

RedBus is an Indian travel agency that specializes in bus travel throughout India by selling bus tickets throughout the country. Tickets are purchased through the company’s Website or through the Web services of its agents and partners. The company also offers software, on a Software as a Service (SaaS) basis, which gives bus operators the option of handling their own ticketing and managing their own inventories. To date, the company says they have sold over 30 million bus tickets and has more than 1750 bus operators using the software to manage their operations.

RedBus previously ran its operations from a traditional data center by purchasing and renting its systems and infrastructure. In addition to the expense, several logistical problems evolved from this arrangement. The biggest problem was that the infrastructure could not effectively handle processing fluctuations, which had a negative impact on productivity. Additionally, the procurement of servers or upgrading the server configuration was an extremely time-consuming endeavor. Over time, RedBus realized that a better solution was imperative—a solution that offered scalability to handle the company’s processing fluctuations. RedBus looked to Amazon Web Services (AWS) for a solution. After testing the AWS solution on a small application for several months, the travel agency determined that it was very workable and convenient. Although redBus was quite enthusiastic about the on-demand instances and variety of instance types, several other features cemented the company’s decision to migrate completely to AWS. These features included the ability to easily manage access to servers through security groups, the easy-to-use, self-service management console, the concept of Elastic IPs, and superior support.

By scaling up and down dynamically based on the load, we maintain performance as well as minimize cost. With the time savings that the IT and development staffs obtain from the AWS solution, AWS gives us an overall cost benefit of about 30-40%. The travel agency anticipates expanding the AWS solution to include Amazon Simple Notification Service (Amazon SNS) and Amazon Simple Queue Service (Amazon SQS) for monitoring, alerts, and intercommunication.

> BMW:

The BMW Group is using AWS for its new connected-car application that collects sensor data from BMW 7 Series cars to give drivers dynamically updated map information. BMW Group is one of the leading manufacturers of premium cars and mobility services in the world, with brands such as Rolls Royce, BMW, and Mini. BMW built its new car-as-a-sensor (CARASSO) service in only six months leveraging Amazon Simple Storage Service (Amazon S3), Amazon Simple Queue Service (Amazon SQS), Amazon DynamoDB, Amazon Relational Database Service (Amazon RDS), and AWS Elastic Beanstalk. By running on AWS, CARASSO can adapt to rapidly changing load requirements that can scale up and down by two orders of magnitude within 24 hours. By 2018 CARASSO is expected to process data collected by a fleet of 100,000 vehicles traveling more than eight billion kilometers.

> Environmental Monitoring Solutions (EMS):

Petroleum retailers in Australia are improving the performance and safety of their service stations with an AWS IoT–enabled solution called Fuelsuite from EMS. EMS specializes in solutions that provide petrol retailers with performance data gathered from sensors located around petrol stations. Fuelsuite uses AWS IoT Device Management to control the edge devices collecting petrol station data, processes the data with Amazon EC2, and schedules messages to and from the edge devices by using Amazon SQS.

> NASA:

The NASA Image and Video Library provides easy access to more than 140,000 still images, audio recordings, and videos—documenting NASA’s more than half a century of achievements in exploring the vast unknown. The architecture includes Amazon SQS to decouple incoming jobs from pipeline processes and Amazon Simple Notification Service to trigger the processing pipeline when new content is updated.

Conclusion:

In the above article, we study the Amazon SQS and the case student of the Top companies that are used Amazon SQS. Hope you understand.

Keep Learning, Keep Sharing!!