Embarking on a Data Modernization Initiative - Traditional BI Platform versus a Modern Data Platform and Adoption Strategies

Going digital starts with more than just desire. Enterprises today may want to design new experiences, introduce digital services, and even embrace a digital-first business model.

The need for insights within an enterprise is no longer limited just in the support of decision making – today, insights are essential in order to power consumer experiences and create product/service differentiation. Data scientists, business users, developers, ecosystem partners, analysts, power users, and casual consumers all want access to clean, related, pertinent data NOW. The idea of restrictions and waiting to get access to data resources does not jive with a digital-first mentality

Many digital initiatives don't get off the ground due to lack of data, poor data quality, or inability to embed insights into apps.

Something has to change – The old way of managing a data estate and enabling the business to drive differentiation does not work. This has prompted many companies to embark on initiatives to modernize their data architecture that enables digital initiatives to come alive.

What is a Modern Data Platform?

A Modern Data Platform is a combination of a set of data architecture principles, data governance processes, a new way of working when building data products, instituting a data-driven culture, and the adoption of cloud technologies that bring these principles and processes to life rapidly.

ETL, Data Warehousing, Reporting, and Dashboarding have been around for a very long time and business intelligence technologies have always dominated enterprise spend. So, what is different now, and how does a Modern Data Platform approach contrast with a Traditional BI Platform?

Contrasting a Traditional BI Platform with a Modern Data Platform

The best way to contrast the two is by analyzing core data and analytics workflows. To do this we will use Productive Edge’s data and analytics workflow framework as shown below.

We will contrast patterns and practices for each of the workflow steps identified in the framework above across a traditional BI platform and a modern data platform. The goal is to tease out the core differences in architectural patterns, mindset, and approach between the two. The list is not meant to be a this vs. that.

There is a place for both a modern data platform, its principles, and a traditional BI platform as part of your data modernization journey.

As you can see above, the differences in approach and patterns are quite different. From rapid innovation in data and analytics technologies to the increasing value of an enterprise's ability to harness data to the rise of citizen developers and to the core need to enable self-service, there are many reasons why a move towards a modern data platform is appealing.

Some questions most companies grapple with as they launch their data modernization initiatives are:

• How to design the most optimal modern data foundation?
• What should my maturity level be across each of these workflow components?
• What do a hybrid traditional BI and modern data platform architecture look like?
• How to create a data-driven culture?
• How / When / Which components to transition to the cloud?
• How to build a roadmap that delivers iterative value to the organization?

Modern Data Platform Adoption Strategies

There are various factors that influence how an enterprise balances the adoption of modern data platform with existing investments in traditional BI platform. Some of these factors include:

• Cloud adoption and readiness
• Culture shift in how data products are built and managed
• Maturity in data governance
• Skills gaps
• DevOps practices maturity
• The speed at which advanced analytics need to be enabled
• Overall speed and maturity of digital initiatives and their dependency on data products

Depending on overall enterprises maturity, there are ways to embark on a modern data platform journey without having to launch a mega big bang project as shown in the image below:

1. The first option adopts modern data platform practices principles and bring them to on-premises or private cloud over time. This could mean adopting DevOps practices within the current traditional BI environment and working on data governance policies and processes or even building a data lake environment that can be lifted and shifted to the cloud at a later time
2. The second option starts putting in place modern data technologies like cloud data lake, data virtualization technologies, enabling a cloud-based ML Sandbox while the traditional data warehouse and data marts continue to provide current needs for reporting and dashboards
3. The third option makes foundational investments like DevOps, data governance and data lake using modern data platform practices, platforms and technologies that are aligned and prioritized with new digital initiatives

There is no right answer on the best path. As I mentioned, the path you choose depends on many factors that I have discussed above. The need to act remains.

A few videos as primers:

• Data Governance in a Modern Data Platform
• DataOps