My data consistency heatmap (part 1: DBaaS)
One cannot understand today’s world without learning history at school. Likewise, in my opinion knowing the historical course GAFAM services take proves invaluable in understanding their long-term "holistic" purpose and making educated decisions to integrate them into an Information System. Agile roadmaps deliver so fast that it is not an easy task, though. Wouldn’t it be great to see the University of Oxford establish a Chair called something like Agile Professor of Public Cloud History any time now?
Like the bold counterpoint in Ravel’s Piano Concerto for the left hand, Cloud Service Providers often kick off a service (the movement) based on seemingly irreconcilable design patterns (the two parts of the counterpoint), then they progressively converge towards delivering a richer customer experience (the composition). I think it is especially clear for database as a service (DBaaS).
This memo has no pretense at being authoritative in databases decision-making, it is a simple dump of the hottest topic in my personal heatmap when it comes to finding the proper architecture design for a DBaaS. And as of today, my top concern happens to be driven by consistency, but chances are it will be different in six months’ time.
Why consistency? Why not availability, scalability, security, durability, reads throughput, writes latency, ...? All these criteria are valid and must be accounted for with wide ranging priorities that depend on the business requirements and the SLA the DB will have to meet, but consistency and durability are trademarks of storage services, the subject of today's article. And since durability harks back to consistency in so many aspects, I feel I'm left with only one choice.
There are many consistency models out in the wild, so to make this article tractable let us consider only two broad, self-explanatory types: immediate and eventual consistency.
- AWS is the giant of eventual consistency: it started off circa 2007 with DynamoDB, the amazing NoSQL store which, along with the help of sound business decisions, allowed to find a sustainable tradeoff around the CAP theorem and let amazon.com scale and become the huge business success it is today. It progressively widened the range of databases types on offer, and went as far as devising its own high-throughput SQL relational database (Aurora). AWS also added a strong consistency option to DynamoDB that I have not tested but that completes the picture.
- Azure (Oracle, and others) are the giants of immediate consistency: Microsoft started off 3 decades ago with the honorable SQL Server which became pervasive within major corporations around the world and ended up being reincarnated into the much revamped Azure SQL. When Azure came out, Redmond engineers designed Azure Table, a highly successful, cheap, strong (within region) and eventually consistent (across regions) NoSQL database. Later on came DocumentDB, which evolved last year into CosmosDB, a technological marvel that lets you choose the consistency model which suits your needs best. Incidentally, the CosmosDB Table API is an Azure Table on steroids.
Interestingly, based on their historical (yes,history. Again…) know-how, both AWS and Azure manage to keep their own silver bullets that you should absolutely keep in mind when making architecture patterns and design choices. Two spring to mind:
- AWS has the DAX, a feature optimized for DynamoDB. As it names implies, the DAX makes your NoSQL database throughput really soar. Like most AWS managed services, it's dependable (within the limits of eventual consistency) and easy: you just fire and forget.
- Azure has reliable collections, the epitome for fast strong consistency (if you are a .NET developer, at least for now). Seeing the huge potential of reliable collections in a serverless world, and with the advent of (the still unfinished) Service Fabric Mesh, Azure architects and product owners have done a great job at disentangling the serverless customer logic from the stateful fabric logic. A stroke of genius that will pay back for sure.
I think both the DAX and reliable collections should be taken very seriously, otherwise you may well reinvent the wheel: a wheel with corners...
In part 2, I will focus more on the 'tomorrow' side of consistency and see how both types will help IT architecture beyond the public Cloud.
Other recent articles that you may like and that you should read anyway ;)
- The Public Cloud coming of age
- An opinionated view of the Blockchain (security architect tutorial #3)