7 Habits of Effective CDOs - the Data Driven Architecture

I've been working for 30 years in Technology and Digital Transformation, with the past 20 focused on Data, and in that time I've heard all of the jargon, Reporting, Business Intelligence, Visualisation, Data Warehouse, OLAP, Column, Graph Databases, Data Lake, Data Lake House, Data Mining, Advanced Analytics, Data Engineering, Data Science, Machine Learning, Artificial Intelligence etc etc etc.

Having seen multiple iterations of similar technologies, and quite a few new disruptive ones, my philosophy has always been to embrace Technology Driven Change. To do that requires an Architecturally Driven Approach, as frankly to use a Car analogy it's no good buying the latest turbo charged or for that matter Electric engine, unless you have the chassis, wheels and instruments to support it. Likewise, I saw through my career so many new systems and transformations attempting to deliver, with no thought on Data and or the Reporting required to actually exploit and leverage the new platform.

At the heart of the Intelligent Business is an Architecture which goes beyond managing all of your data, to realise the vision of an architecture where Data masters Process and Technology to serve people. I'd like to explore that in thinking about two of today's hot 'Data Tech' buzz words are Data Mesh and Data Fabric:

• Forrester analyst Noel Yuhanna was among the first individuals to define the data fabric back in the mid-2000s (and we we had a similar conceptual model at Capgemini, where I led the Data practice there).?Data fabric is an architecture and set of data services that decouples from any physical implementation. It provides seamless access across multiple clouds, data centers (core), or even edge systems (such as Internet of Things devices, local machines, or even mobile devices)
• Data mesh aims to solve many of the same problems i.e. the difficulty of managing data in a heterogenous data environment–it tackles the problem in a fundamentally different manner. Data mesh encourages distributed groups of teams to manage data as they see fit, albeit with some common governance provisions, whilst Data Fabric seeks to deliver a single virtual management layer for all data.

These two concepts are being used to address Data Management, but I would like to think of them as how they address a truly Data centric business model. The picture at the top of this blog is my Conceptual Model for a (Intelligent Business) Data Driven Architecture. Logically it builds on the Data Fabric approach, and is, I believe is critical for businesses going future. Data Mesh will certainly allow functions and divisions of a business to manage their own data, but ultimately for an organisation to be sustainably competitive it needs to be able to leverage all of its data and manage that outside of all the enterprise business systems (whether Cloud or on premise).

By transforming your Technology Estate and migrating to a Data Driven (Micro Services) Architecture in the Cloud, you can realise massive efficiencies:

• Reporting is no longer a big focus. Instead Data Democratisation makes the same data available to all of your managers and they can leverage appropriately to support Management Decisions at the Board, Marketing, Sales, Production etc. The focus of most of your managers will be less on understanding what happened and instead using data to create scenario plans and models that allow the business to look forwards. Predictive by design (and no longer managing the business looking through a rear view mirror)...
• Critically, Data is not just managed for Decisioning, it is managed to full and drive the core business systems that operate your business. AI and Machine Learning models you design increasingly provide the logic that drives your business, and you and are less dependant on logic provided in 'industry standard' software. As more logic moves into your Data estate, your monolithic business process systems (CRM, ERP, SCM etc) can be dramatically downsized and become what Technologists call ‘Micro Services’ – at a fraction of the cost.??Most software today is bloated and massively over engineered. Micro Services delivered the essence of key features and functions, ensuring that your business buys and pays for what it needs, whereas today we typically use less than 20% of the software we've paid for. Having a Cloud based Data Model, that is controlled and owned by your business and which can orchestrate the activities of all of the software partners you rely upon to serve your Customers, your Suppliers and business partners has to be the most effective model for business going forward.
• Your Technology Estate is increasingly sitting, not in your premises, but in the Cloud, and your Technology Suppliers are managing the Costs and Risks around that. Whilst for some it makes sense to migrate everything to Azure, GCP or AWS, personally I am a fan of a heterogenous model. Firstly, because as we've proven in the last 25 years, it's almost impossible to get all of your data in one place! Secondly, and more importantly, if your data is your crown jewels, do you want it to be controlled and managed by one company (whether Microsoft, Google, Amazon or ANOther...). Having a Fabric where you can manage the data across multiple clouds and premises makes eminent sense to make this work!

This is of course a massive change for most larger and mature businesses BUT if executed corrected can deliver the agility, flexibility and customer centricity of a start-up, to a mature multi-billion pound/dollar/euro business. Inevitably, it's a journey, which has to be driven around Value, AND this architecture is driven around the biggest source of Value - your Customers. If you want to compete with or partner with Amazon you need to start moving away from the Digital Models that have amounted to 'lipstick on a pig' (a nice web-site or App, hobbled by arcane legacy systems and bad data) and instead embrace the Intelligent Business/Data Driven Architecture.