The Design of an Event Store
The topics “event-driven architecture” “event stream processing” and “event sourcing” have enjoyed quite a buzz as of late. While the concepts are not new, it would seem that only now the software engineering community is beginning to appreciate the power and flexibility of building autonomous, loosely coupled systems that intelligently react to events, rather than being told what to do.
Now that our industry has gotten on board the event-driven bandwagon, some of us are starting to look beyond our usual building blocks — Kafka, Pulsar, and NATS Streaming — to address the concerns of long-term persistence and intelligent retrieval of events. We are, of course, talking about an event store.
An Approximate Definition
When contemplating event stores, the first question we should ask ourselves is: what exactly is an event store?
As it stands, a canonical definition of an event store does not exist… yet. Everyone has a different understanding of what an event store ought to do, although most practitioners have come to agree that an easily queryable, long-term persistent store of event records is probably in order, given our ongoing investment into this architectural paradigm.
A “store of event records” hardly sounds convincing on its own. Realistically, something as straightforward as this could easily be accomplished with a database, or even Kafka itself. (And other event streaming platforms, for that matter.) Ostensibly, practitioners wouldn’t have coined a term unless it stood on its own merit. So, we turn our attention to the “queryable” part.
We know that the one thing that makes a database powerful is its ability to be queried. There are lots of ways one might query events. Recall, an event is an immutable record of some (presumably) interesting action that occurred at some point in the past. So, for starters, we might want to query event records by a time range. Next, we might want to filter events by certain attributes or relations. For example, “find events that refer to customers X and Y”.
By observing a stream of discrete, chronologically ordered events, we can accurately reconstruct the state of the entity to which these events apply. At the very least, we should be able to reconstruct its observable state. (Where “observable” implies a characteristic, whose change is fully described by an accompanied event record.) Reconstruction of observable state is the very essence of event sourcing. Wouldn’t it be convenient if an event store was somehow augmented with an ability to enact event sourcing?