August 2023: The Age of Data Services

Welcome to the third edition of the Komprise Intelligent Data Management newsletter! ?We cover new ways for IT managers to be more productive managing enterprise data and storage to dealing with ever-changing compliance issues, working with departments on data strategies and understanding the new requirements for data management and AI.

?Last month we tackled the heady topic of cloud cost optimization. Can you move to the cloud, leverage its innovations, and still get your ROI and more? With the right data management approach, the answer is: Yes, you can! You can read it here. Learn more about Komprise, a SaaS for unstructured data management and mobility here and follow us on LinkedIn.?

This month’s newsletter dives into the emerging trend of data services in unstructured data management.

These data services include protecting and archiving data (standard fare for storage pros) but also understanding and managing ever-changing compliance requirements, managing data lifecycles, working with department data stakeholders on cost efficiency strategies and yes even deleting data when it's no longer needed. Data services responsibility also means making the right data easily available to end users and tools to reduce time to market for new research soon and analytics initiatives.

Why data services and why now?

For decades now, IT infrastructure teams have been responsible for evaluating, procuring, installing and maintaining high-performing storage appliances from companies like Dell EMC, HPE, IBM, NetApp and Pure Storage. These days, enterprise IT organizations are managing storage in the data center, at the edge, and in the cloud. They are likely using more than one storage vendor and multiple technologies for different workloads, geographies and needs such as backup, data protection and disaster recovery.?

Data—especially unstructured data like documents, images and video—is massive and growing exponentially today. Storing and backing it up consumes at least 30% of IT budgets in most organizations.

Unstructured data contains the insights to compete, operate more sustainably, keep employees and customers happy, save money, boost productivity and unleash cool new products to market.

In that vein, IT managers and storage architects are responsible for managing data independently of where it is stored. Moreover, executives want them to deliver a host of data services across the business. ?Now, with AI at the heart of business and technology plans, the efficient delivery of data services is even more imperative.

In this article in IT Pro Today, Komprise cofounder and COO Krishna Subramanian covers the topic of data services in-depth.

In this excerpt, she explains the benefits of adopting a data services approach in IT:

• Holistic visibility and granular search of data across multiple storage systems and clouds;
• Analytics and insights on data types, data growth, key metadata characteristics and usage for more accurate storage and lifecycle decisions;
• Automated, policy-driven actions based on that analysis;
• Reduced security and compliance risks from better visibility and automation;
• Full, native use of data wherever it is stored, especially in the cloud;
• User self-service access to support departmental and research needs for data storage, management, and AI workflows;
• Greater flexibility to adopt new storage, backup, and DR technologies because data is managed independently of any vendor technology.

Here are a few industry examples of data services in action:

Medical device maker:?A company with regulated products must regularly answer questions from auditors about its data: What is it, who owns it, and how is it used? Ensuring that data is being stored and protected according to various regulations such as HIPAA is imperative to avoid large fines and penalties. The company is also working to bring in "shadow" data from remote sites so that IT can ensure permissions and other protections are up to date. Using a solution that indexes all unstructured data is the key to accomplishing these critical compliance tasks.

Life sciences:?The central IT team at a global pharmaceutical firm has been tiering cold data to the cloud but now wants to give its research teams the ability to identify and tag their project files for later use. This approach flips the dynamic from IT having to police data storage to providing an analytics-based data service to the business.

?

Retail:?A national retail conglomerate went through a divestiture, shuttering a few of its brands and standardizing its IT infrastructure. The process resulted in a large quantity of zombie files from technologies no longer in use, such as Microsoft .pst files. A data services toolset allowed the organization to quickly find the unwanted files across all storage and delete them. With a simpler, more streamlined data environment, the company is in a great position to grow in its next stage of evolution.

Entertainment:?The storage director at a large Hollywood studio was investigating why certain shares were taking an excessively long time to back up. By running analysis on the data, the director discovered many old files belonging to a handful of users. Once the users were notified, they deleted the files, which improved backup cycles..

Data services require new data metrics

Delivering timely data services means you’ve got to know as much as possible about the data. Storage teams have traditionally measured metrics like latency, I/O operations per second (IOPS) and throughput; employees tend to go nuts when things run slowly.?

But now IT folks need some additional data-centric metrics to keep things running well and cost-effectively in a hybrid cloud infrastructure. This VentureBeat article goes into detail on these new metrics.

Here are a few to consider:

• Data growth rates (overall and by department, location or storage technology)
• Top data owners, data types and sizes
• Costs of data storage, by department, geography, technology etc.
• Predicted data storage costs
• Percentage of “cold” files which haven’t been accessed in over a year
• Sustainability metrics such as duplicate data reduced, data stored by vendor (some vendors are greener than others)

?

With data services, IT infrastructure professionals change too

IT job roles are always in flux as technology changes so rapidly and storage architects/engineers will also evolve to deliver data services. Already, these individuals have been under pressure in recent years to understand much more about cloud computing. In some enterprises, experienced storage engineers are moving to cloud architect roles while IT generalists/junior cloud engineers inherit their responsibilities.

This is a challenging time for IT organizations in a hybrid model as there is still significant on-premises expertise needed, especially in Network Attached Storage (NAS). Either way, the IT employees managing the storage function will need new skills beyond managing the storage hardware, including:

• An understanding of the concept of data services, such as facilitating secure, reliable governance and access to data;
• Making data searchable and available to business stakeholders for applications such as cloud-based machine learning and data lakes;?
• The ability to frequently analyze and interpret data characteristics;
• Development of data management plans which factor in cost savings strategies and business demands to create new value from data;
• Regular interaction with departments to create and execute ongoing data management processes and plans.

How Komprise Supports Data Services

Komprise has a data-first approach to unstructured data management. While that sounds a bit redundant, we have developed our solution to be an independent data management platform. That means IT managers can make and execute decisions about data independently of the storage technology. They can move data to where it needs to be and users and applications aren’t disrupted in any way. Researchers and data scientists have cloud-native access to any data that has been moved to the cloud – which means they can easily use their data in any cloud AI or big data applications. That means faster, frictionless time to analysis: and who doesn’t need that these days?

At the heart of our data services is Komprise Deep Analytics and the Global File Index, which is like a Google search engine for your data. This five-minute demo gives a taste of how it works:

IT and authorized department users can search for files across all storage (on-prem or in the cloud), by file type and location or directory (aka., all video files in the Atlanta site or owned by Marketing department) metadata tags (indicating that the file contains for example, IP, PII, geographic, demographic or project data), age and time of last access.

This means users can crunch the data in an endless number of ways:

• Are users in India regularly accessing data stored in North America? Could I move those files closer to them so they get better performance?
• Are there anomalies--like a large number of files modified in a particular directory, possibly indicating ransomware--that require further investigation?
• Which department or user is growing data the fastest and why?
• Are there files still around from defunct applications that can be deleted?
• Are any employees storing files against policy, such as personal photos and videos?
• Where are all the files from project X, so I can move them to a cloud data lake?
• How much duplicate data is in my department?

Last Words…

Data services is still a relatively new concept in IT infrastructure circles.? We’ll be following this trend closely in the coming months and would love to get your feedback on how data services, AI, unstructured data management and data storage developments are affecting your organization. Comment on the post or send a note to: [email protected].

?

?

?

?

?