Is Microsoft Fabric just another analytics platform? Why should organizations take Fabric seriously for their AI transformation?

There is tremendous amount of excitement with Microsoft in the recent past and more specifically, in the analytics space with the imminent launch of Microsoft Fabric. A complete analytics platform designed and built for the era of AI.

We have seen endless innovations in the past 2 decades in the data world and being in this space for long enough, I thought, I would do an unbiased rationalization (yes, I promise, and I will make the case) of this technology. In order to accomplish that, I decided to look at Fabric from:

"What's in it for the customers?"

Is it just another analytics platform? Another addition to the endless list of innovations in the past 2 decades?

Let's rationalize.

Data world has been abuzz since 2000s and we have seen endless inventions, reinventions and refinements in technologies which continue to push the boundaries of our ability to understand people, things and processes with a singular aim of improving efficiency.

Organizations have always been performing these analytics for over 30 years. The change and challenge now are how fast and quick organizations wish to get these insights and even more importantly, how they can take these insights to action.

So, its abundantly clear that it all comes down to one factor: "efficacy".

Now, let us understand what are the downward drags which prevent every customer from achieving this "efficacy" at scale.

1. Data

We all know that every organization has been doing analytics for the past several decades. This analytics was confined to the structured data from their applications like ERP, CRM, SCM et al and the boundary of analytics was limited to "operational reporting" focused on yesterday, week, month and years.

Since 2000s, with emergence of companies like Yahoo, Google and all other social media platforms, the data types exploded in variety and volume. In addition, came the IoT and edge in late 2010s. We all have heard that 90% of the world's data has been created in the last 4 years and 80% of that data is unstructured. Factually, "click steam data" is the fastest growing amongst all data types.

For long until few years ago, every customer was focused on solving the challenge related to "how can we bring all these data to analytics?". I can confidently state that we have solved that challenge!

2. Analytics

With abundance of data available, organizations understood that operational reporting alone would not be sufficient to make sense of the data and that, they needed to rely on data science for exploration and hypothesis, real time analytics and near real time analytics to get insights sooner and take them to action.

Quite simply, the boundaries of analytics quickly expanded beyond operational reporting.

It is my belief that, more than 50% of enterprise analytics workload in less than few years would be real time analytics and AI applications consuming data from the data platform.

3. Data Infrastructure, Governance, and Operating Model

Now, this world is labor intensive and has always been lagging behind the needs of the organizations. Three reasons:

(i) Organizations which were in their own comfort zone of having all data in "one place with data retention policies" found themselves inadequate to the realities of:

(a) Seeing their data landscape multiplying faster beyond one on-prem to multiple sites across multiple geographies including multiple cloud platforms

(b) The paradigm shifts in data retention policies making way for "data is gold" ideology.

(ii) The operating model of serving data:

IT organizations were always at the center of this operating model to identify, clean, and transform all the data to serve to the businesses. This model came under stress when organizations started to do real time analytics.

(iii) Data Estate - Infrastructure Complexity

The customers' data estate evolved over a period of time. It kept up with the phase of needs of the organizations and the customers kept adding one technology over another as and when they came to solve a specific problem.

What started with a simple database and storage now have proliferated to an average of 40+ technologies to make up for a data estate in an enterprise.

With all these drags came, every organization is struggling to find value (insights) from their data and to take them to insights, just in time.

Now, let us candidly analyze the factors which can solve these challenges and improve efficiency for organizations to get value from their data.

Solution 1: One Place for all Enterprise Data, can it be?

We know the promise of Data Lake with the advent of Hadoop.

The idea that every customer data can reside in one place cost effectively appealed to all of the customers and Hadoop became a phenomenal success in a very short time in the mid 2010s. Quickly, the excitement faded with enormous challenges to maintaining these huge infrastructure on-prem and the never ending need to keep updating the technology at the growing phase of innovation in the Hadoop world.

Now, every customer understands that it's impossible to have all data in one place.

So, what's next?

How about, customers can have most data in one place, we respect customers' choice to keep their data wherever it makes sense to their needs, but provide super easy access to these data sources without compromising the enterprise requirements like security, governance et al?

How about we provide data virtualization capabilities without adding another infrastructure complexity to customers?

Fair solution?

Solution 2: Dealing away with multiple data formats, can it be?

With data types growing beyond structured data, and every application and edge environment producing their own proprietary data, we know the amount of effort and time consumed in data transformation.

How about we automate this data transformation process and spare the customers from the most labor-intensive data processing work?

We know 70% of the time in analytics is just data preparation and bringing automation to this process can move the efficiency needle in a big way.

How about we allow the customers to keep all of their data in one single format, most importantly, in an open format?

Solution 3: Do away with proprietary compute engines, can it be?

With evolved needs for data integration to engineering to data science to real time analytics, we know the customers run multiple compute engines which require the data format to be in a specific format.

This requirement forces every customer to ingest and re-ingest data multiple times just to cater to the needs of their compute engines.

How about we make all these compute engines to use the same open standard format?

Solution 4: Infrastructure simplicity and focus on "Time to value" than system integration.

The lessons of Hadoop world taught us enough that customers should be focused on "data value creation" and not doing system integration work and spending time to keep up with these technologies with updates and refreshes.

How about we provide a seamlessly integrated platform that customers can sign up for in seconds and focus on their work in minutes?

No data extensive sourcing, preparation, transformation, system integration. Just drag and drop the data sources (mostly) and get on with work?

There can be several other factors which can add to the efficacy cycle. But I am convinced that these 4 solutions will move the needle to another level of radius.

With AI disrupting the data landscape, removing latency from the data systems is weighing on the top of the mind for every customer. Latency will be the key metric moving forward for every application.

AI is never about leveraging APIs to operationalize few use cases. Every organization understands with humility that,

• Their data needs to be in order and
• To create a sustainable AI based organizational transformation, they need to have embrace a healthy cycle of "insights from analytics on their own data" powering the "automation through AI".

If customers were to focus on solving for latency and efficacy with the above, I strongly believe that Fabric can move the needle for customers in a big way.

Fabric empowers customers with abilities to,

• Sign up in seconds,
• Ingest and store data in one place,
• Create short cut to external data sources seamlessly,
• Automate the data preparation process, and
• Achieve better ROI for their use cases with superior performance and shared compute pool for workloads.

Fabric can dramatically reduce the number of technologies in a data estate to a single unified platform.

Finally, customers can now just focus on their work.

"Data value creation for their organization" with efficacy at scale!

So, is Microsoft Fabric just another analytics platform?

If you there is any further doubt, please watch Arun and his team at MS Ignite. For now, I will keep this article confined to my personal opinion!

Marcio A. Goncalves Cesario , Zia Mansoor , Arun Ulag , Jessica Hawk , Mandana Javaheri , Harsha Konduri , Binaka Shah Sankaran , Vaishali Chawan , Mario J. Vargas Valles , Nevenka Scott , Charlotte Dick-Cleland , Ege Onelcin , Shreya Ghosh , Sharif Islam , Swapnil Khopkar , Gina Khan , Kelly Rogan , Brian Maslowski , Harman Cheema , Marianne Roling , James Serra , James Lee , James Caton , Simran S. , Michele Fisher , Tamer Farag , Karlien Vanden Eynde , Eric McChesney , Ronald Chang , Thasmika Gokal , Dan Houdek , Vibhu Ranjan , Dinis Couto , Srinivasan Venkatarajan , Soren LAU Jarnail Singh , Jason Hermitage , Jason Langlais , Eric REICH , Marc Chemin , Glenn Finch , Francesco Brenna , Sumeet Parashar, PhD , Elayaraja Eswaran , Anil Nagaraj , Bret Greenstein , Will Johnson , Anna Stamatelatos , Owais Hashmi , Anthony Mattas , Rima Semaan , Nermeen AlAhmadieh , Sandeep Basu , Kim Manis , Bogdan Crivat , Priya Sathy , Jason Langlais , Amol Shah , Jason Pereira , Alan Grogan , Emerson Gatchalian , Laura Lee ,