The Five Commandments of Big Data Cloud Migration

The cloud is here to stay. Data cloud migration, however, is proving vexing. According to McAfee, 97% of organisations use public or private cloud services. Big data is starting to jump the on-prem/cloud gap too. IDG found that 41% of enterprises are already migrating storage/archive/backup/file servers to the cloud, and 21% plan to do so in the coming year.

These numbers are great news for cloud adoption rates and cloud big data vendors…unless you look at the cup as half-empty. Because then you see the 59% who are not migrating, or the 79% who do not plan to do so in the coming year. And you wonder, with all the benefits of housing big data near the apps that produce and consume data, why are so few enterprises jumping in head-first?

The answer, it turns out, is simple: it’s really hard.

Why data cloud migrations get bogged down

To migrate big data to the cloud using existing paradigms and tools demands serious technical prowess, and seriously deep pockets. Take a look at Lyft’s AWS bill and you’ll understand why so many companies shy away from even trying.

Here’s a news flash: migrating big data to the cloud requires finesse. Brute force just doesn’t work – which is why 62% of big data migration efforts are harder to complete than expected or fail.

IT stakeholders tend to overlook the fact that physics are still involved, even in the digital universe. Current methodologies try to move data in bulk via the digital equivalent of a shipping container – ignoring the fact that data is not a static cargo. It’s in constant flux. Business doesn’t stop during migration. Data keeps flowing into existing, on-prem storage even as you move to the cloud. This makes maintaining data integrity during migration really complex. And since common workarounds tend to fall short, this is where many migrations bog down.

Avoid the pitfalls: The five commandments of data cloud migration

To avoid the pitfalls of big data cloud migration, shy away from brute force options. Adopt a more finessed and sophisticated approach, and keep in mind the following five commandments of big data cloud migration.

COMMANDMENT 1: Know thy data

Before you start a migration project, make sure you truly understand your data sources. Create a systematic process to identify data sources that are migration candidates. Then, for each source, ask yourself:

1. Is this data live and/or actually used for anything?

2. How high is its migration priority on a scale from 1 to 10?

3. How much effort will migration involve on a scale from 1 to 10?

4. What are the costs and risks associated with maintaining the data during and after migration?

COMMANDMENT 2: Thou shalt not lift and shift

To be blunt, lift and shift is dead. This methodology represents the ultimate brute force cloud migration (see above), wherein applications and associated data are “lifted” from on-premise environments and “shifted” to the cloud. The problem is that on-prem environments and cloud environments are apples and oranges. Sure, they’re both round, but who wants an orange pie? Just because your architecture worked great on-prem doesn’t mean it will make sense in a distributed computing environment.

COMMANDMENT 3: Seek not savings

Migrating big data to the cloud offers many advantages: it’s a productivity turbocharger; it’s a deep source of unprecedented business insights; it’s a whole new way to know customers and trends and get up close and personal with data. It’s not, however, a way to save money. You can, and should, control migration and operational expenses – but don’t expect to gain the massive advantages of cloud-based big data for a lower price tag.

COMMANDMENT 4: Invest in thy people

With limited IT resources and already busy teams, most SMEs lack the in-house bandwidth and expertise required to manage a cloud migration. Outsourcing is a solution, of course, but consider in-house training as an investment. Distributed computing requires a specific skill set and your cloud investment shouldn’t just be in services but also in your staff.

COMMANDMENT 5: See not data as a liability

With the tremendous effort and complexity of making an effective move to the cloud, IT stakeholders, staff and executives alike can begin to view organisational big data as more liability than treasure. Don’t let this happen. Don’t store just what you need and throw away the rest – there’s hidden value there. Keep your data the way you keep your important stuff when you move to a new home – have your garage sale after you move, not before.

The bottom line of data cloud migration

With big data migration to the cloud, the bottom line is that muscle doesn’t cut it. Lift and shift and other brute force migration techniques leave question marks surrounding data integrity, and ultimately raise migration overhead — sometimes prohibitively. The new wave of migration approaches is gentler, more staged — and far more user- and cost-friendly. With the right tools, your cloud adoption strategy doesn’t need to take forever. It can still happen this year.