Eventual Consistency in system Design

Fred would often receive PUT requests from a client and inserts the data into the database and replies back with the id.

Fred’s life was simple until a new data auditing service started asking for any data that Fred receives to go to it as well.

This creates two distinct problems.

Problem #1

Imagine inserting the data into the database took around n milliseconds, which meant that in the first scenario the client making the request would take n to receive the id and finish the request. Let’s also assume that the auditing service requesting the data takes around m milliseconds to consume it, so after adding that service the client making the request now takes n+m to finish the request.

Since we’re building architecture that scales, this time will keep increasing as we add more services.

Problem #2

It’s a fact of life that sometimes requests fail for any number of reasons: the client is not ready, validation failed, too many requests..etc..etc.. So what would happen if one of the requests fail, i.e. if the data was inserted into the database but didn’t get through to the auditing service

We can’t tell the client that the request failed because it kind of didn’t and if we try to re-insert the data, the database would complain (we can get around that by upserting but we’d be bending a business rule)

Eventual consistency?to solve the problem

Way #1 database first

The request is limited to inserting the data in the database; after it’s inserted Fred tells his client that the data is in and the request ends there but the data’s journey doesn’t. The database triggers the MQ and sends it the data that it has just received, the auditing service picks up the data and consumes it without affecting the reliability or the time it takes to complete the request.

In that method, we guarantee that the data is in the database by the time the request ends but we can’t guarantee that it is in the messaging queue.

Way #2 messaging queue first

In that method, we guarantee that the data is in the messaging queue but not in the database. The API is also responsible for creating the identifier for the new data.

You’d use that method if having your data in the messaging queue is more important than having it in the database.