10 Key Signals of Change That Will Re-define the Way We Look at Data in 2022
This is a slightly long article, so if you want to skim through, then you may read the summary report here.
We have been talking about “data-driven enterprises” for almost a decade. Yet, we are still far from being truly data-driven. Organizations with well-established data initiatives are also struggling to unlock value from their data.
Over-promised & Under-realized Value from Data & Analytics Investments
A 2021 report by NewVantage Partners shows proliferating investments in Big Data & AI over the last few years, despite the pandemic. Over 30% companies surveyed reported that their investments in Data & AI increased due to the pandemic, while almost 60% reported unchanged investments.
What is noteworthy is that while investments in Big Data & AI remain high, proven business results continue to lag.
There is a burning need to bridge the gap between data investments and value. While a cultural shift, executive sponsorship, data literacy and company-wide commitment will act as drivers, here are some key key signals of change that I believe will shape the way we look at data in 2022.
Signal 01: Rise of the Data Mesh
For many years now we have believed that centralization of data will solve all our problems. However, this model doesn’t scale too well. The centralized data warehouse or data lake becomes a bottleneck between data producers & consumers. While this worked well for simple domains and smaller number of data sources & consumption use cases; it failed for organizations with rich domains, fast increasing number of varied sources and consumers, frequently changing user requirements, and a need for agility & faster time-to-value. The answer is the Data Mesh.
Simply put, data mesh works on decentralization & distribution of responsibility. Producers (from each data domain) publish their data products themselves (in a trusted, accessible, consumable way) and consumers fetch data & integrate on a need basis in a self-service manner. This also requires a federated & global governance model that enables interoperability of data. This approach avoids the need for multiple, large-scale projects & helps realize value faster while reducing risk by eliminating the “middleman” between data producers & consumers.
Many companies are seen adopting data mesh principles without necessarily calling it so. This adds solid credibility to these principles and prove that the mesh is more than just another buzzword. And we will certainly continue to see multiple similar architectures (whether or not they are called “data mesh”) in 2022.
Signal 02: Living on the “Edge”
First let’s start with what is edge? Edge is not equal to IOT. It’s a topology that integrates centralized & distributed architectures. It refers to a range of networks or devices at or near the user, and have the ability to talk to the cloud. Content Delivery Networks are a fantastic example of the edge where content is hosted as close to the users as possible for faster delivery.
Distributing data & analytics to the edge reduces latency and enables more near real-time responses & use cases, as well as autonomous behaviors. It allows for data pre-processing and/or filtering at the edge producer before it is moved to the central data store to optimize transfers. This helps distribute the load across the ecosystem and provides improved robustness. However, complexities will also increase with data on edge creating new avenues for innovation around edge data management & governance.
Edge data management has even emerged on the rise in Gartner’s hype cycle for edge computing, 2021.
Shifting data & analytics to the edge has multiple applications and will open multiple new opportunities for scaling D&A capabilities and extending impact to different parts of the business, and different geographies too.
Signal 03: The Hybrid, Multi-cloud Imperative
According to Gartner, a staggering 80% of user of cloud data management services use more than one cloud service provider (CSP), while 47% have their data management both on premise & in cloud (one or more CSP).
There are so many varied use cases from data distribution or data replication across cloud & on-premise, storing data on-premise but running ML models on the cloud, cloud bursting for meeting spikes in storage or compute demands, edge data etc. It allows for a best-fit architecture with fit-for-purpose services from appropriate CSPS and/or on-premise tools.
So, it is clear that data is shifting to multiple clouds/hybrid cloud but data management on hybrid & multi-cloud is not straight-forward. It requires a specialized approach & a new outlook to data management to achieve success in the hybrid, multi-cloud world.?
Signal 04: Everything Ops
The bandwagon started with the advent of DevOps – bringing development & operations teams together to accelerate the development lifecycle. But slowly, this concept has begun to touch almost every domain in technology, especially the world of data. Today we are talking DataOps, MLOps, ModelOps, PlatformOps, AIOps, DevSecOps and even NoOps. The ones, I believe, are critical to the data world are:
It is becoming more and more critical to explore the world of XOps, and have XOps capabilities embedded deep into your data supply chains to make them leaner & drive incremental value from data.
Signal 05: Architecting a Sustainable Future with Data
Sustainability is no longer optional, but a mandate. Data is growing fast, and so is the demand for storage, data center workloads, and internet traffic. According to a Statista report, the number of hyperscale data centers worldwide has doubled from 2015 to 2021.
领英推荐
Storing & processing data sustainably is crucial to achieving any organization’s broader sustainability goals. Accenture’s sustainability innovation lead, Sanjay Podder, highlights 4 key areas of improvement in a recent article:
All key cloud service providers are working towards reducing the carbon emissions from their data centers. However, the burden of sustainability cannot lie on the shoulders of cloud providers alone. Every enterprise must make a shift to sustainable data & analytics practices and establish sustainability programs to meet their “net-zero” goals.
Signal 06: Shift from Big Data to Small Data
Small data, as the name suggests, is capable of using modeling techniques that require lesser quantity of data but are still able to provide valuable insights. An HBR article says that “many of the most valuable data sets in organizations are quite small - kilobytes or megabytes rather than exabytes; but because this data lacks the volume and velocity of big data, it’s often overlooked.”
They conducted a 12-week experiment and came up with three human-centered principles to get organizations started on their small data initiatives:
76% of executives agree that organizations need to dramatically reengineer the experiences that bring technology and people together in a more human-centric manner. Small data is truly a close collaboration of humans and AI.
Signal 07: Best of both worlds with mainframes & modern data systems
Mainframes are thought of as old, legacy systems. Yet, 70% of Fortune 500 companies and 92 of the world’s top 100 banks operate on mainframes. Mainframes handle a staggering 30 billion business transactions every day. The performance, resilience and security that mainframes provide remains unparalleled to date. Research and Market data predicts that by 2025, the global mainframe market will increase by $2.9 billion. But, mainframe systems remains ridden with challenges – uncontrolled & growing total cost of ownership owning to the expensive MIPS (ranging from $0.27K to $8.9K per MIPS), lack of skilled mainframe talent, tightly coupled applications, and limited advanced analytics.
83% C-suite executives want to maintain the best of legacy while moving to new technologies. Most of the mainframe modernization work so far is focused on applications while completely overlooking a critical element – data. ?It is time to leverage the analytical horsepower of modern data systems (scalability & economics of cloud) while making the most of existing investments on mainframes.
Signal 08: The Power of Digital Twins
A digital twin is a digital representation for physical objects and is fast growing across industries. 87% of executives agree digital twins are becoming essential to their organization’s ability to collaborate in strategic ecosystem partnerships. Organizations are creating digital twins for all their key assets, and data is at the core of it all.
Imagine infusing real-time data from digital twins with simulation, machine learning and human intelligence to make smarter, better decisions. It can gravely improve operations, maintenance and even help evolve new business models. Today, twins exist not only for inanimate objects; we are even talking about human digital twins. The possibilities are endless with this phygital world, and can definitely prove to be a competitive advantage to adopters.
Signal 09: AI for Data
Enterprises are now treating data as a competitive asset or more rightly their “intellectual capital”. But it is critical to optimize the data supply chain to make the most of this asset. There’s an increasing amount of data about data available today and even more possibilities to run analytical models on this metadata.
Multiple use cases are already in place such as intelligent data quality management, smart data classification & profiling, smart regulatory compliance, etc. Performing analytics and building AI models on this metadata can translate into improved data quality, faster time to value, and reduced operational expenses among other benefits.
Signal 10: Beyond Dashboards – Decision Thinking & Analytics-as-a-Product
How often do we find a business analyst frustrated about having spent months of effort building a dashboard that goes unused? Does that mean there’s a problem with the users or the dashboard? The true problem lies in the misalignment between the two. Purpose-driven product thinking for analytics will help you go beyond dashboards, and create the right “analytics products” for your “consumers” (or users) to make the right “business decisions”. Try answering questions such as:
Final Thoughts
While some other horizontals continue to remain in focus such as security, automation, data trust, modernization, customer experience, self-service, unified governance etc., these signals of change bring a multi-faceted prism of possibilities. They truly hold the potential to re-invent the way we look at data.
All of these signals hold potential to transform or even disrupt the world of data. Now is the time to catch these signals to change your data outlook in 2022 and take a step closer to being truly data-driven.
Every big journey begins with a small step.
ACKNOWLEDGEMENT: Special thanks to Pragya Sharma for her guidance & support in authoring this article.
Disclaimer: The views, information, or opinions expressed in this blog are solely those of the individuals involved and do not necessarily represent those of their employer and its employees.
References
Ritu, very nicely articulated. Thanks for sharing, and keep it going ??
Author, Design Thinking Coach, Dean-Business Design, REDX, Innowe, NISP @Weschool
3 年Nicely compiled. Keep ?? ??
Data Architect | Sales Engineering at Snowflake
3 年Very well written article Ritu. Had been insightful to read through these changes in paradigm and how enterprise data would shape up in future. Cheers!
Mainframe Modernization Consultant and Cloud Architect at Accenture DACH | IBM Champion 2025
3 年Nice write up Ritu. Very good insights and you have touched upon important aspects of Data. Mainframes for sure will play an important role in Data World with 80% of corporate data still on it. One question thou Data mesh is more of Pub Sub offering (Kafka?).