“Is The Data Model Finished Yet?”

Naturally this title caught my attention.

Intrigued, I read on.

The article argued that with the advent of Big Data, we no longer need to waste effort putting together often complex and time consuming data models. In this transformed data landscape, everything can be developed rapidly to meet a specific set of requirements. As soon as the next set of requirements arises, if the previous development does not fit them, it will be jettisoned and replaced.

Part of me wondered at this thinking; can the world’s organisations now really operate in this way because the data landscape been so utterly transformed by Big Data?

I remembered back to similar articles I had read over the years. Each repetition of this death knell has occurred with the advent of a major new paradigm on the data landscape. Examples over the years include; the relentless advance of COTS[1] products, the Internet, Agile development, the Cloud and, of course, the Big Data Lake.

But still the data model has survived.

My mind started to sift the evidence that I have personally witnessed. Big Data certainly has had a significant impact on organisations that require data to support their operations, and therefore also on their data models. But I can see no indication that even this seismic shift in data collection and analysis has caused these core organisational definitions to die out.

In fact, for many organisations, a key outcome of adopting Big Data has been the exact opposite; it has resulted in the realisation of their importance. The explosion of technical innovations that have transformed data usage by organisations, has fundamentally altered the way that data models are required to support them in this data-rich environment. Whereas in previous times, they may have been restricted to being viewed as an ‘unwelcome’ but necessary part of development, they are now being recognised for what they truly are; a definition of an organisation's operational lifeblood.

There is now a realisation that data models allow an organisation to ‘know thyself’.

Arguably, they are now more important than ever. It is only with a full and agreed understanding of the ‘What?’, ‘When?’ and ‘How?’ of an organisation’s data structures and flows, that we can contemplate plugging COTS products together, implementing in the Cloud, ingesting data into our Data Lakes, or reporting across the Enterprise system landscape. In the last decade, data models have made the transition from being ad-hoc and limited in scope, to becoming a central pillar of the Enterprise Data Architectural landscape.

Thus, a beneficial by-product from the adoption of Big Data is the realisation that its true benefit cannot be delivered without being able to correlate the meaning from Big Data analysis with the organisation’s Master Data Domains. This has driven the focus for organisations to be able to bring their Master Data under control and thus the processes that manage it.

So, I took what I could from the article, finished my sandwich, and resolved that it was now time to start writing the book that I had always wanted to. Its purpose would be to:

• Explain the power of data models
• Describe the easy steps required to define and quality assure them and
• Define the processes that harness their power to deliver maximum benefit

I finished the book in late 2016. It is entitled "The Data Model Toolkit - Simple Skills To Model The Real World" (ISBN - 13: 978-1782224730) available here in the UK and here in the US.

It is one in a growing popular series including the "Enterprise Data Architecture - How to navigate its landscape" also available from all the usual online stores including here in the UK and here in the US.

[1] Commercial off the Shelf