Kafka Event sourcing in Event Driven Architecture

What is Event Sourcing ?

Event Sourcing?is ensuring every change to the state of an application is captured in an event object, and that these event objects are themselves stored in the sequence they were applied for the same lifetime as the application state itself.

so instead of storing just the current state of the data in a domain, use an append-only store to record the full series of actions taken on that data.

What is Event Driven Architecture ?

A microservice architecture is an architectural pattern that structures an application as a collection of small, loosely coupled services that operate together to achieve a common goal. Because microservices work independently, they can be added, removed, or upgraded without interfering with other applications.

An event-driven architecture?uses events to trigger and communicate between decoupled services?. An event is a change in state, or an update, like an item being placed in a shopping cart on an e-commerce website.

What is kafka echo system ?

Please have a look at :

Conencting the dots : Kafka echo system & Event sourcing

• Simply , you can use Apache Kafka borker ! it maintains an immutable sequence of events that multiple applications can subscribe to so that Multiple producers can subscribe to multiple topics, and publish events to multiple partitions. on the other hand Consumers on the other hand are divided into consumer groups, consumers in the same consumer group can only read from different partitions. withing each topic , messages are guratnteed to e executed in order.
• In more complex case , Producres can be a of other popular systems like ElasticSearch,PostgreSQL, BigQuery, MySQL.. etc .. Kafka Connect — a framework (APIs) and runtime for plugins that integrate Kafka with external systems.it can be deployed in two modes. either standalone mode or distributed mode.These Connectors come in : Source Connector — used when sourcing data from another system (the source) and writing it to Kafka. Sink Connector — used when sourcing data from Kafka and writing it to another system (the sink).

Long story short is that events to change state of the application is recorded to a db for example ,then a source connector add it to a topic on Kafka topic to be consumed later to communicate with other microservices .

Conencting the dots : Kafka echo system & Event driven Architecture

For the decoupled microservices . Instead of checking for new data, micor services just listen to a certain event and take action.

decoupled microservices here are the consumers of the Apache kafka topics

unlike messaging-oriented systems, events published in Kafka are not removed as soon as they are consumed. They are removed once a specific amount of time has passed.

Conencting the dots : Why Kafka ?

1. Durable, Persistent Storage
2. Scalability
3. High performance

Use case : Uber - One of the Largest Kafka Deployments in the World

Refrences :

[1] https://dzone.com/articles/microservices-event-driven-architecture-and-kafka

[2] https://hevodata.com/learn/kafka-event-driven-architecture/

[3] https://nexocode.com/blog/posts/event-driven-architecture-examples-in-logistics-apache-kafka-to-handle-supply-chain-network/

[4] https://learn.microsoft.com/en-us/azure/architecture/guide/architecture-styles/event-driven