An Introduction for Event-Driven Architecture on AWS
by Santiago Arango, Software Architect at Growth Acceleration Partners
In this edition, you will find a very accessible and effortless way to understand event-driven architecture, how it can be used, what its purpose is, and the advantages of adopting this kind of design in your products. Let's dive in.
Event Driven in the real world
Let’s keep it simple at first. In our daily lives, we live through events. Events are nothing more than actions you can execute at any time, but you might not get a result in real time; you may have to wait for some time.
Let’s take buying a car as an example. You head to the dealership and choose the car you like that fits your requirements. Now, a couple of things could happen here. The dealership might ask you for an advance payment for the car, and then your request goes into a queue with other clients who also bought cars.
Here, things can change depending on many factors. If you buy a standard car with a lot of marketing and sales, you might end up in a bigger queue. But, at the same time, the factories usually produce these kinds of vehicles more often due to the high sales figures. On the other hand, if you buy a premium car with a smaller buyer list, you might have to wait longer for production and delivery since they are more complex and time-consuming.
From this example, we can abstract the action of buying a car as an event. The seller puts your order into a queue, and the factory processes the orders likely in the order they arrived, but they would split the list according to the type of vehicle you asked for. So, as you can see, the factory is a consumer because they pick up the events and process them. However, even though they are consumers, the factory can also be a producer.
After the factory completes the production of the car, they create another event to notify the transportation area to pick up the cars and move them to the target destination. Eventually, the car arrives at the dealership, triggering another set of events. For example, one event could be preparing the car by cleaning, washing, and waxing it. When this event ends, the seller will reach out to you, and then you will get your brand-new car!
As you can see when you trigger the first set of events, many other events happen one after the other, and you may not be aware of all of them. What you care about is getting your brand-new car when the time comes. However, it doesn’t mean all events must happen one after the other. For example, when the company ships the car to the location, the dealership may do a bunch of things at the same time, such as creating invoices, charging you with the rest of the payment, paying insurance, registering your car with the local authorities as the owner, among other tasks.
Base participants in an Event Driven Architecture
Now, let’s translate some basic concepts from real-world examples. As you might have noticed, we have events, consumers, producers, and queues. Let’s give a brief description of each one of these parts so we can mix them into an architectural design.
Events
As the name suggests, an event is something triggered by a particular action. This action can be started by the end-user, other software components, or events that trigger other events. For example, when you tap your phone to open an application, you produce an “On Click” event. The software on your phone will pick up the signal produced by the tap and continue to open the application you selected.
It’s important to note that sometimes you may have to wait for a response after triggering an event, which means it acts as an asynchronous event that will give the response sometime in the future. For example, when you place an order for a car, the manufacturing request may be taken later. This is what happens when you place an order on an e-commerce site like Amazon. You may have to wait a few minutes or even days until they confirm the seller has the product and then try to charge your credit card.
Producers
Now, this one is quite simple; the producer is the entity, or maybe even other events that trigger events. As an example from the dealership, when you pay and reserve the car, you are the producer of an event of buying a car, so you produce that message by making the purchase.
In a software example, you are also a producer when you buy something on Amazon; for example, you produce an order that the right services will process.
Consumers
Consumers are one or more entities that listen to produced events. Usually, software events are wrapped up in structures that can be easily understood between applications. For example, JSON or XML formats are quite popular since most programming languages or PaaS platforms support them. The consumers’ jobs can vary according to the system’s requirements. They can transform messages into something different and send those messages to other consumers, or they can be at the end of the road and do something specific with those messages.
This is where queues come into play, as we mentioned earlier. A queue system is another software part whose main job is to take those messages, store them for some time, and then ensure the proper consumer or producer receives them. The consumer can wait until the message is read and processed. Whatever implementation is used, queues implement robust and resilient strategies to ensure any message that arrives is processed, and in case of failure, it notifies the proper components.
AWS Services for Event-Driven Architectures
In this section, we are going to describe some AWS Services components that help build event-driven architectures. Then, we will move to a chapter to deep dive into benefits, and finally, we will build the dealership example with components!
AWS Lambda
AWS Lambda is a serverless computing service that allows you to run code in response to events or triggers. It is a perfect fit for an EDA because it enables you to create small, independent functions that can be triggered by events, such as changes in a database or messages in a queue.
Amazon SNS
Amazon SNS (Simple Notification Service) is a fully managed messaging service that enables you to send messages or notifications to many subscribers or endpoints. It can be used to decouple your application components and let them know of events that occur in other parts of your system.
Amazon SQS
Amazon SQS (Simple Queue Service) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. It can be used to transmit any volume of data at any level of throughput without losing messages or requiring other services to be available.
Amazon Kinesis
Amazon Kinesis is a fully managed streaming data platform that enables you to ingest, process, and analyze real-time streaming data at scale. It can be used to capture and process events from a variety of sources, including IoT devices, social media feeds, and weblogs.
AWS EventBridge
AWS EventBridge is a fully managed event bus service that enables you to build event-driven architectures using events from AWS services, SaaS applications, and custom applications. It can be used to simplify the integration of your application components and automate workflows.
Step Functions
AWS Step Functions is a managed service provided by Amazon Web Services that enables users to build serverless workflows to coordinate distributed applications and microservices. With Step Functions, users can create complex workflows with conditional logic, error handling, and retry functionality through a visual interface. The service models applications as a state machine, where each state can execute AWS Lambda functions or other services. Additionally, Step Functions integrates with other AWS services and offers features such as parallel execution, nested workflows, and real-time monitoring, allowing users to build reliable, scalable, and fault-tolerant applications.
Benefits of AWS Event-Driven Architecture
Okay, now that we understand the basic parties involved in event-driven architecture (EDA), let’s dive into the technical details and explore the advantages of using EDA in your product.
EDA offers immense value to software by supplying scalability, lower latency, resiliency, and cost minimization. Why? Because EDA enables you to build highly available products that can withstand technical failures in some components without fully affecting the entire system (unless there is a massive failure that affects services by region).
For example, let’s say you’re ordering a car online. You choose your car with all the details you prefer and then place your order. Later that day, you receive an email confirming your order was placed correctly and accepted by the dealership. Now, you need to make your payment. However, at that exact moment, the service that generates invoices is down.
But that doesn’t stop you from making your payment. In an EDA system, the payments and invoice services are not connected. Instead, the payments service adds a new message to a queue, and the message sits there until the invoice service takes it and processes it. Sometime later, when the invoice service is back online, you will get your invoice, and the process of getting your new car will continue.
Now, if the same scenario happened in a traditional architecture where the user must wait until the system processes everything and the invoice fails, you would have to call the dealer; they would have to verify your payment and then ask someone to process the invoice properly. This is because the components are tightly connected as a single process.
Dealership Sample with Event-Driven Architecture using AWS Services
In the image above, you see an example of how an EDA system can interact. This example uses AWS Services, like Lambda, SQS (Simple Queue system), RDS (Relational Databases), SNS (Simple Notification System), and step functions.
Let’s split it into smaller pieces:
Dealership Sample with Event-Driven Architecture using AWS Services — After Order
In the earlier section, we triggered a step function. The great thing about step functions is they allow you to create states for your application, which is perfect for our case. If we follow the process of delivering a car after you place an order and the dealership confirms it, it can be summarized as follows:
Now all these steps can be easily adapted to our application using EDA. As you see, this is going to be a perfect scenario for step functions because we will have several states for the vehicle from now on.
Each one of the states would be set by a user. This will be done via an event according to the result of the previous one. Let’s draw it as part of the Step Function!
Step functions to change states
Let’s take a look at what we have on Diagram C. Here, we can easily summarize all the step changes into a single design by changing something according to the current step. One thing that will not change during the entire process is that someone will be using an application to see the status and update it to the next step. For example, a manager might be the one to see the orders in the queue for the different assembly lines and choose which one should go into production next. When the car is in production, the assembly line boss would be responsible for updating the status. The same applies to the QA lead, delivery area and so on.
What can change here is the Lambda function titled in the diagram as “Function Update Status.” You can easily see it this way: a Step Function is used to have several states on which actions are triggered, and for each action, you can have something associated with it. You can trigger a Lambda function, add messages to SQS, send notifications via SNS, and more.
This can be taken to mean each step on the function can trigger a unique Lambda function and do whatever it needs to do according to the state. For example, when the car fails the QA process for some reason, maybe they not only want to update the status but also send custom emails to the heads of operations so they are aware of the low quality. Another example is when the car leaves to be delivered; the dealerships receive a delivery note with the cars that are coming their way.
Closing Notes
One small note I want to give to you, my dear reader, is that if you design an application following this pattern, it is simple and easy to scale, especially if you are using services like AWS. If this app is your product and you start selling it, it would be ready to handle one or thousands of dealerships easily. This design also gives you heavy resiliency, using queues, step functions, notification services, and others in the cloud, which give you flexibility if one part of the process fails. The services can come back online later and do their job, unlike standard architecture, where it’s all or nothing.
We keep it simple here, but you could easily integrate services like EventBridge and add one or more queues for the delivery lines to increase scalability and make your design more solid and stronger. But even with a simple connection of services, you could supply a great solution!
And finally, I am Santiago Arango. I have been working in the software industry for 12 years, mostly as a software developer and technical lead of engineering, and now, in my most recent job, as a software architect.
Interested in joining our team? Visit our careers page.