The Data Migration Challenge

Welcome to the latest edition of the Komprise Intelligent Data Management newsletter! ?We cover new ways for IT managers to be more productive managing enterprise data and storage to dealing with ever-changing compliance issues, working with departments on data strategies and understanding the new requirements for data management and AI. Learn more about Komprise, a SaaS for unstructured data management and mobility here and follow us on LinkedIn.?

Last month we covered the results of our annual survey, Komprise 2023 State of Unstructured Data Management.? Read it here .

This month’s newsletter covers cloud data migrations. Moving data around is the thankless job of the IT or storage manager. It's necessary to keep data in motion to take advantage of newer, better, lower-cost storage technologies and/or to start new cloud initiatives such as an AI projects. Yet this is a risky endeavor if not done well--and so much can go wrong, from missed deadlines, to lost data, expensive consultants to fix issues and other issues after the migration if performance isn't what users expect and cost savings aren't what IT expects.

According to Flexera State of The Cloud 2023 , organizations continue to invest heavily in the cloud despite the prevalence of the hybrid cloud environment and a slowdown on cloud spend early this year:

• 63% of enterprises are spending more than $1.2 million in the cloud infrastructure.?
• Nearly 30% are spending more than 12 million.
• As well, 45% of respondents said?economic uncertainty has little impact?on cloud usage and spend.

Enterprise cloud strategies still seem to be going strong, yet data migrations are complex.? Top challenges include technical feasibility, rightsizing instances and analyzing costs of using different options between on premises and the cloud, according to Flexera.? When we work with our customers on migrations, issues pertaining to network and security configurations often come into play.

That’s why planning is critical!

Data migrations are a team sport. Having a checklist of what you need to know before you go will help ensure a faster, more effective cloud migration that meets your goals.

1) Define Data Storage Sources and Targets

Develop a clear plan that reviews where you’ve come from and where you’re going. Understand point A and point B and don’t wait until the night before a migration to ensure this is clear.

• Cloud native apps often need object storage. Are you prepared for this?
• Cloud NAS has matured. What is your plan?
• Do you still have requirements to keep data stored on premises?

2) Unstructured Data Migration Rules & Regulations

Rules may cover areas such as retention policy, legal hold and disaster recovery. Regulations are typically set by a governing body and often involve fines if not followed, such as: HIPAA, SOX, GDPR.

• Partner with your legal and compliance teams.
• Get the data owners involved. Put it on the teams that care about the data so it’s not just on the shoulders of the storage team to establish the right data protection and unstructured data management strategy.

3) Know Your Unstructured Data

When you do proper data discovery, you’ll understand your workloads and speed bumps. Do you have large files? Small files? Hundreds or thousands of shares?

• Dual vs. mixed protocols: What is your permission strategy and how will this affect your migration?
• Simplify and standardize: ?Just because you’ve been doing things a certain way in the past doesn’t mean it’s the right way in the cloud.

4) Smart Data Migration. Know Your Topology

Define your path and understand what you have and will need for bandwidth, latency, firewalls, etc. when it comes to your hybrid cloud migrations. A core objective is how to avoid bottlenecks ahead of time. Bring in security and network teams.

5) Before You Migrate Data: Test, Test, Test

Test sources and targets, topology and migrations. Consider whitelisting. Know the limitations of your current file storage systems and your future targets. The iPerf tool is helpful to measure storage performance and to understand how long it will take to move data across the wire.?

6) Free Tools: Cloud vs. Enterprise

There are many free tools to help with migrations – from manual scripting and scheduling to MD5 checksums and audit logs to multi-protocol support.? These tools have limitations compared with commercial enterprise solutions which offer centralized configuration and management, automation and reporting features and can tier and archive data. Dig into the Komprise Elastic Data Management white paper to learn more.

7) Data Migration Communication Planning

Don’t worry about overcommunicating when talking about migrations as people tend to panic when they can’t find their files. Send concise emails with red/yellow/green status updates. People want to hear the positives. Avoid the blame game and celebrate the wins.

What does Komprise offer for file and object data migrations?

We just announced updates to Komprise Elastic Data Migration product which is part of our full Intelligent Data Management platform. ?In 2022, we announced Hypertransfer , which delivered a massive performance improvement of 25 times faster SMB file migrations. In October, we added?capabilities for three specific use cases to give enterprise IT organizations more flexibility to adapt to changing needs:

Pre-loaded migrations: Also known as a “jumpstarted migration” or “bootstrapped migration”, this is a type of migration where an organization has performed the first iteration (the “baseline copy” of all the data) using a tool or mechanism other than Komprise (e.g., NetApp SnapMirror, Dell PowerScale SyncIQ). They then want to use Komprise to perform the ongoing synchronization of the source (which is a live file system, where users are adding, modifying, and deleting data) with the destination, and then finally perform cutover. The goal of a pre-loaded migration is the same as a standard migration, i.e., to “make the destination an exact copy of the source”, except the destination will initially be non-empty.

“Zero downtime” or “warm cutover” migrations: A warm cutover migration will minimize the downtime experienced by end users and applications at the end of a migration, when the source is set to read-only to allow Komprise to copy over the last changes made on the source. By allowing end users and applications to use the destination (especially to add data) even while the final iteration is running, Komprise enables a “warm cutover” or “zero downtime” migration, providing the least impact to end users and applications.

Consolidation migrations: Consolidation migrations enable multiple sources migrating onto one destination (e.g., to consolidate hardware, sites or data, due to acquisition, merger or to re-organize existing data). A simple consolidation migration may consolidate data into different subdirectories on the destination. Users can configure and run multiple standard or pre-loaded migrations into subdirectories, in parallel, as needed.

Check out the short demo:

Last Words:

Cloud data migrations are becoming commonplace--but not easier. Getting it right depends upon truly understanding your data and its usage patterns, user and compliance requirements, technical infrastructure and costs. Keep following our newsletter as we continue to delve into pertinent topics for unstructured data management in the months to come.

Comment on the post or send a note to:

[email protected]

?