Marketers Ask: What Can Hadoop Do That My Data Warehouse Can’t?

This is the 1st post in a 5-part series, “A Big Data Cheat Sheet: What Marketers Want to Know.” This spin-off series for marketers was inspired by a popular big data presentation I delivered to executives and senior management at the SAS Global Forum Executive Conference earlier this year.

In April 2015, I was given the opportunity to present An Executive’s Cheat Sheet on Hadoop, the Enterprise Data Warehouse and the Data Lake at the SAS Global Forum Executive Conference in Dallas. During this standing-room only session, I addressed these five questions:

• What can Hadoop do that my data warehouse can’t?
• We’re not doing “big” data, so why do we need Hadoop?
• Is Hadoop enterprise-ready?
• Isn’t a data lake just the data warehouse revisited?
• What are some of the pros and cons of a data lake?

In this 5-part blog series, I will answer these questions again, but this time, with a focus on how these big data technologies are impacting (or can impact) the customer experience, and what marketers can do to take advantage of this data playground. Let’s get started!

Question 1: What can Hadoop do that my data warehouse can’t?

Here’s the short answer: (1) Store any and all kinds of data more cheaply and (2) process all this data more quickly and cheaply.

The longer answer is: [Please excuse me as I step up on one of my big data soapboxes (I have many!) to address this question.]

I’m here to tell you that big data is not new. Yet, with all the hype these last few years around these two little words, you’d think we’ve discovered the Holy Grail. Let me share with you the dirty little secret about big data: it’s just data—the same data we’ve had for decades.

They say that 20% of the data we deal with today is structured data (see examples in orange boxes above). I also call this traditional, relational data. The other 80% is semi-structured or unstructured data (examples in blue boxes), and this is what I call “big” data.

Are any of these blue-box data types new? Of course not. We’ve been collecting, processing, storing, and analyzing all this data for decades. What we haven’t been able to do very well, however, if at all, is mix the orange- and blue-box data together.

So here’s what’s new: We now have the technologies to collect, process, store, and analyze all this data together. In other words, with Hadoop, we can now mix-&-match the orange- and blue-box data together – at a fraction of the cost and time of our traditional, relational systems. You can’t do that with your data warehouse.

[I’m stepping off my soapbox now.]

Why this matters

Big data technologies like Hadoop take the “360-degree view of the customer” concept to a whole new level. Let’s say you want to provide your customers with an omnichannel experience, so that no matter how they choose to interact with you, you’re right there with them. It’s possible with data. The diagram above includes 25 sample data sources, many of which contain customer data. What if you could tie these data sources together to provide your customer with a satisfying and even fun experience? 

Consider this scenario: One of your loyal customers posts on Facebook that she’s going shopping at one of your stores today. You know that she just purchased a pair of pants online last week, and that her abandoned online shopping cart has a few cute tops in it to go with the pants. She goes to the store, the retail assistant is able to identify who she is and brings out the tops she abandoned online to try on with her new pants. But since your customer isn’t wearing her new pants, the retail assistant knows which size pants to go grab. Then while shopping, your customer gets a 25% off coupon delivered to her smartphone—good for today only.

All creepiness aside, this retail scenario is not as far-fetched as you may think. This is what mixing-&-matching your customer data will allow you to do. With Hadoop, not your data warehouse.

Key takeaways for marketers

Before you go bust down IT’s door and ask them to install Hadoop so that you can have a better 360-degree view of your customers, please understand that this is easier said than done. Whereby these big data technologies make mixing-&-matching your data possible (which is a huge feat in itself!), be aware that the tools themselves are still maturing. You will need technical assistance—from IT and developers, internally and externally—to get started with Hadoop.

But it’s not just about the technology. I strongly encourage you to follow these three steps if you want to be successful with Hadoop:

• Identify the business issue. Don’t “do Hadoop” just because everyone else is doing it or because it looks good on paper or it’s cheap to install. Do Hadoop if it helps address or solve a real business issue for your organization.
• Get executive buy-in before—not after—you get started. Don’t embark on a big data project without executive support. Even successful projects have been shot down because they couldn’t get executive support and/or they didn’t support corporate strategies.
• Develop a multi-player plan. Don’t do Hadoop, or big data for that matter, alone. It’s not a single department play. Big data projects require multiple players from the business, IT, and executive management.

Many companies eager to jump on the Hadoop bandwagon have missed these three steps, and guess what they have to show for it now? Abandoned Hadoop installations.

Don’t be one of those companies.

Originally written for and published on the SAS Customer Analytics blog.