Unstructured Data and Its Heavy Price

Welcome to the latest edition of the Komprise Intelligent Data Management newsletter! We cover new ways for IT managers to be more productive managing enterprise data and storage to dealing with ever-changing compliance issues, working with departments on data strategies and understanding the new requirements for data management and delivering AI-ready data. Learn more about Komprise, a SaaS solution for unstructured data management and mobility and follow us on LinkedIn. ?

This month’s newsletter covers the economics behind managing unstructured data.?

We will look at the cost of unstructured data, why it’s so expensive for the average enterprise IT department to manage, and some tactics to reduce storage costs and waste.?

Your Unstructured Data Is Costing You – A Lot?

The top priority for data storage in the next year is cost optimization (54%), according to IT directors and executives in the Komprise 2024 State of Unstructured Data Management. Almost half of enterprises in the survey are storing more than 5PB of unstructured data and nearly 30% have more than 10PB.

Storing and backing up billions of files across hybrid cloud environments is expensive, exceeding 30% of the IT budget for more than half (55%) of respondents this year.??

This is not surprising since unstructured data—any data that doesn’t live in a database, such as user documents, texts and chats, audio, video, images, machine data--continues to expand rapidly due to the accelerating pace of digitalization, SaaS, AI, IoT and more. Yet as we all know, this unstructured data is the golden ticket to AI, ML and so much more. Analyzing and understanding unstructured data assets can help you develop stronger customer relationships, more targeted and effective products and services, streamline operations, increase employee productivity and help tackle global, societal problems.?

Do You Know What Your Data is Costing You??

?But first, we need to understand this data. Through analysis and segmentation, organizations can make better, more cost-effective decisions for the management of data and create the right data foundation for generating new value from unstructured data.?

In this article in Dataversity, our Field CTO Benjamin Henry Henry walks through the issues and challenges:?

CXOs this year have witnessed a rollercoaster economy amid plenty of turbulent events – from ongoing inflation affecting consumer spending to large stock market swings, major overseas conflicts, and the uncertainties of an election year. Not surprisingly, the economic forecast remains murky at best.??

Yet here’s the glitch: Nobody wants to be left behind in the latest wave of technological disruption. Chief officers are looking at AI with a hard eye. They want to leverage AI magic soon for both operational efficiency and competitive advantage. They know that AI will eventually require notable investment across people and technology.?

If unstructured data is the fuel to power AI, IT leaders must ensure the organization’s data is in the best shape possible. This requires that IT teams clean up the data mess and get control of the unstructured data that is fueling AI and ML. A looming barrier is that today’s data estate incurs high costs to manage, impeding the budget for AI.??

Here’s why unstructured data is so expensive:?

?

1. Unstructured data, which makes up at least 90% of all data being generated in the world, is heavy and hard to manage.? For a large enterprise with multiple petabytes of data, the data storage and backup bill can easily run in the millions. And 60% of IT leaders say that their costs will go up next year, according to the Komprise survey.?
2. Data hoarding contributes to the problem.?It’s more common to keep all the data year over year than to institute data deletion, or even data archiving policies. Enterprises often retain unstructured data for decades because it contains useful information such as customer insights, potential research intelligence, and machine learning training data for the future. Audits and compliance requirements also support the long-term or never-ending storage of data. Yet with today’s data growth rates, it’s no longer sustainable economically or from an energy standpoint to keep data forever.?
3. Data storage is only 25-30% of the total cost.?Most of the expense of storing unstructured data lives beyond primary storage. IT teams need to protect data with backups and replicate it for disaster recovery and that means creating multiple copies of data to store, manage and secure. Yet most data is not mission critical or active and can be archived at a lower cost and without making multiple copies.?Read more here.
4. One-size-fits-all storage:?The previous point leads to this one; organizations too often store all or most of their data on expensive?network attached storage (NAS)?technologies when most of it doesn’t need that level of performance and availability. Years ago, enterprises didn’t have enough data to worry about where it was stored. Today, though, there are many classes of data storage on-premises and in the cloud that help organizations store data in the right tier at the right time to save money. Placing “cold data” that hasn’t been accessed in a year or more to lower cost, secondary storage can save anywhere from 60 to 80% a year on annual data storage, backup and DR costs.?
5. Lack of visibility:?The ability to save on data storage and implement the cost-effective strategies outlined above is impossible without insights on the data. This includes its volume and rate of growth, storage costs, types and sizes of files, top owners, and data usage trends. This lack of visibility also brings data governance and compliance risk: You cannot protect your data if you don’t know what it is or where it lives.??Learn about Komprise Analysis.

?

Benjamin goes on to offer a few tactics on how to reduce these costs without sacrificing performance and access for users:?

If you can get metrics on your data from the data center to the cloud, create data management policies for different data sets, and automatically move stale or cold data to lower-cost archival storage such as in the cloud, the savings can be astounding.??

On a 4PB NAS environment with a 30% year-over-year growth rate,?your enterprise could save over $2.6 million or more annually with the right cold data tiering and/or archiving strategy alone.?

?

?Unstructured Data Cost Savings Decision Tree?

?Here are three ways to model potential cost savings from unstructured data management:?

?

Cold data potential savings?

--How much data is rarely-used or cold??

--How much you can save by archiving it, based on your own cost model???

How much data is orphaned such as from ex-employees???

--How much is this costing us today???

--How can we save either through archiving or staged deletion???

--This also improves security posture and reduces risk.??

Duplicates potential savings??

--How much data might be potentially duplicates???

--Should we have a process to work with data owners to reduce these??

?

Video: How to Reduce Data Storage Costs with Randy Hopkins?

?

Having insights on your unstructured data is critical for cost savings measures in many ways. As just one example, 22% of unstructured data is unnecessarily replicated because organizations simply don’t know what they have or how to find it, according to IDC’s report: “What Every Executive Needs to Know About Unstructured Data.” Insights on data are also critical to help generate new value from the data.

Only half of an organization’s unstructured data is analyzed to extract value, and only 58% of unstructured data is ever reused more than once after its initial use, according to IDC.??

?

To learn more about unstructured data cost optimization, read the white paper.

Last Words?

Unstructured data costs are not poised to decline or flatten anytime soon. The time is now to better understand your data using tools that provide global visibility into data across storage, drill down capabilities into file shares, dashboards to model new storage plans and automated policies to continually move data to the right place at the right time.?

You can subscribe to our blog to receive new posts in your inbox and check out what's new by visiting our Resource Center.??

?Comment on the post or send a note to:??[email protected]??