Event Driven Architecture

Introduction:

• An architecture style that leverages asynchronous processing to trigger and respond to events within system.
• Systems don't need to know about each other to share information and accomplish tasks.
• A maintainable, sustainable and extensible model for building complex, distributed applications where scalability, responsiveness and flexibility are critical.
• It is widely used in modern application development, including microservices architectures, real-time data processing systems and complex event processing systems.

There are producers which produce the events and consumers which listens to these events and respond to these events. There is possibility that one consumer can subscribe to multiple event streams. In the bigger picture responsibilities can overlap. One system can act as a Producer or Consumer based on the use cases.

It is always a loosely coupled architecture.

What is coupling:

When there is integration between two systems coupling exists.

There are many facets of coupling in Architecture:

In the above example, "Purchase Book" event will go to "order Placement " service" which will trigger "Order Placed" event. There are 3 services responding to this event "Inventory", "Notification" and "Payment". Once "Payment" processing is completed another event "Payment applied" is triggered. "Order Fulfillment" service will respond to this service. "Notification" Service is listening to "Order Place", "Order Fulfilled" and "Order shipped" events. "Inventory" service listens to "Order Placed" event and updates inventory and triggers "Inventory updated" event. "Warehouse" service listens for this event and based on that creates event "Stock replenished". Once all processing is done "Notification" service will trigger event "Customer notified".

There is always a initiating event and then there are derived events. Services can server single function or they can be a major subsystem.

As you set on this journey of Event Driven architecture you may face with some obvious questions.

Things to consider for Event Driven Architecture:

So response to many of these questions takes us multiple patterns for Event Driven Architecture.

These are common questions and challenges you may face when designing Event Driven Architecture.

Following are 8 patterns which are widely used.

For choosing from each of these patterns we may have to do some trade-offs. There are some good things and some bad things. So there are consequences.

Event Driven Vs Message Driven:

It is important to understand the key differences.

There is a difference between Event Driven and Microservices based Message driven systems. In event driven it is notification, publishing it and it does not care who is listening to that notification. .

Event Contract Patterns:

There are two types based on how payload is sent.

Pros and Cons of Data based Contract.

Data based contract gives scalability and performance.

Data based contract may not be a great option to support multiple Systems of record.

With Data based contracts management and versioning will be a challenge

Pros and Cons of Data based Contract.

Data based contract gives scalability and performance. As in above example as "Payment" and "Inventory" services is getting all the data, they don't need to go to the Database to fetch the data and less hits to the database.

Data based contract as data is broadcasted to multiple Services, we effectively have multiple Systems of Records. In this pattern, if Event A is triggered, then immediately, new Event is triggered to change the order. So now because order in which consuming services get the data, it can cause data integrity issues.

With Data based contracts management and versioning will be a challenge. There is possibility that each consumer services needs to consumer different set of attributes from the data contract.

What is Stamp Coupling? It occurs when Services share same Data contract but consume different parts of it. But because of stamp coupling even if other area of Data contract is changed consumer gets impacted.

In above example rate of Producer service is more than consumer service consumption rate. This creates issues with Data based contracts.

Key Based Contract

Key based contract only transmits key attributes and hence payload is small. If consumer needs additional information it can go to the database.

As multiple calls may happen for data to the Database from multiple services, there can be scalability and performance issues.

With Key based contract as all the data requests are served directly from Database, we alleviate issue of multiple systems of records. This will insure data consistency. As it is only key fields chances of stamp coupling are low.

Overall, bandwidth usage is optimized as payload is much lighter and data is requested from the database.

Again, depending on use cases one or combination of these patterns can be used.

Event Forwarding Pattern

This pattern guarantees that there is no data loss while processing.

There is possibility that channel can fail and it is point of failure. Service B can crash and another point of failure. Database can crash and this is last point of failure.

To address point of failures, we need durable subscribers for Service A and data should be sent synchronously. That means data is sent to Topic and Service A waits until it gets confirmation and ack. Channel/Topic will persist those messages. It will guarantee that there is no loss of event/message in transit.

When Service B gets the data from topic, topic will wait for ack before processing next event and until then messages/events are persisted on the topic. If Service B crashes, it can again read from the topic as data will be still there.

If Database crashes, Service B will have committed data and can resend that.

So, producers will not remove the data until it gets confirmation from consumer of successful delivery.

Kafka, Rabbit MQ support persistence of messages/events.

Overall turnaround time can go down because of synchronous processing.

Domain Brokerage Pattern

It is about managing topologies of the brokers. Events need to be housed somewhere. For instance, whether it is Kafka stream, Topic in RabbitMQ or Broker can seat somewhere in the cloud.

Workflow State Pattern

These are patterns which considers state management.

Thread Delegate Pattern

This pattern can be used to guarantee that we maintain, order of processing. These patterns usually have two purposes, how to preserve the message order and then tracking of event processing.

Supervisor Consumer Pattern

This is one of the most common patterns. In this pattern Supervisor is responsible for adjusting amount of consumer. It will listen to channel monitor and determine if current consumer can handle the events. If throughput of the consumer is not enough it may call for more consumers for the help and vice versa.

Multi Broker Pattern

In this pattern capacity or throughput can be significantly increased.

I didn't cover here all the patterns in detail, but point is one pattern may not fit your particular scenario, so you may end up choosing more than 1 patterns and combinations of those to support your performance, scalability, data consistency needs.

#eventdrivenarchitecture #microservices #looselycoupled