Cloud Data Warehousing—Seeing Patterns in the Cloud

There’s an old adage. You wait ages for a bus and then four come along at once. After a fairly quiet decade of “data lake-ism,” a convoy of cloud data warehousing solutions arrived in the market around 2020. Why four different answers to a question that has been asked since the skies clouded over in the mid-2000s?

Readers of “Cloud Data Warehousing—Volume I: Architecting Data Warehouse, Lakehouse, Mesh, and Fabric” may recall my analysis showing that moving from on premises to the cloud requires absolutely no change in the conceptual architecture for data warehousing and only minimal adjustment to the logical level to accommodate the emergence or evolution of a few technologies and the highly distributed and disparate nature of the hybrid cloud / on-premises environment. So, what could possibly drive four distinct patterns for delivering data warehousing in the cloud?

The slightly cynical answer is “marketing!” The cloud data warehouse pattern represents an attempt by traditional warehouse vendors to protect their market share in a straight contest with newer competitors starting anew in the cloud with much the same RDBMS-based solution. The lakehouse pattern protects the market of the data lake / open-source players of the last decade. Data fabric is the punt of the big analyst firms, while data mesh is the consultants’ play on the theme of modern software development methods requiring lots of handholding in the face of absent or immature software solutions. Am I being a bit too skeptical?

“Cloud Data Warehousing—Volume II: Implementing Data Warehouse, Lakehouse, Mesh, and Fabric” (out now) posits that there is a little more to it than the lucid dreams of marketing teams. By elaborating architectural design patterns (ADPs) on a common base for the four contenders, we can see where these approaches differ and where they overlap. The result is a useful foundation for organizations facing the dilemma of how to adapt to the growth of cloud and which solution to adopt.

These ADPs all start for the same logical architecture base but add an additional layer of detail that moves thinking toward the physical level, without needing to delve into the product feature comparison matrices beloved of too many implementation teams.

Over the next couple of months, I’ll be offering high-level views of each of the four ADPs. But there’s no need to wait! Both Volume I and Volume II are now available to order online in hard- and softcopy from both Amazon and from Technics Publications, via the above links, where a 25% discount is available with coupon, TP25.