What is Big Data?

What is Big Data? Rather than define a specific size at which data becomes big, it’s helpful to understand why there needs to be a fundamental shift in strategy toward Big Data and what makes the problem of Big Data different than small data.

Imagine that you have a bathtub full of water, and you need to get the water out to another nearby bathtub. What are the sorts of strategies you could use to move that water? You could get a bucket to scoop it out. You could create some sort of hose to pump the water over, or you could get strong people to lift the tub and dump it wholesale. These are all decent solutions for getting the water out.

Moving up a level in scale, if you had a large swimming pool, what sorts of strategies can you use? The bucket strategy would probably be too time consuming, the pump strategy really begins to shine, and the amount of effort taken to dump a swimming pool out would be tremendous. You might be willing to build a more permanent pump mechanism, power it with electricity instead of a hand crank, and build it out of copper tubing instead of a rubber hose, which can malfunction or break under too much pressure.

So far these two examples, the bathtub and swimming pool, represent the type of problem solving that takes place in the small data space. But what if you needed to move a whole lake of water? How would you solve your problem then?

Once we’ve reached the lake level, we begin to have to define and refine our needs and resources. Do we have to move all the water at once? Do we even need to move all of the water, or just the clearest water? What’s lurking at the bottom of this lake? How permanently do we need to move the water? Are we just migrating the water, or is something being done with it?

At the lake level and beyond (the rivers, the seas, the oceans), we’ve finally arrived at the Big Data space. Our strategies for managing bathtubs and swimming pools simply aren’t going to cut it, or they are going to be costly. We begin to wonder about the value of the effort.

But moving lakes and harnessing rivers and accessing the resources of seas and oceans are the things that civilizations are built on. A dam that utilizes the flow of water, a submarine that can scout out the enemy in open waters, an irrigation system that can turn desert into arable land—these are powerful technologies.

So, too, with Big Data. If you can harness the real time movements of the marketplace, if you can plumb the depths of the market and see what your competitors are doing or what goods your customers are most likely to desire next season, you will have access to some of the most powerful, life-giving information your business can have.  

To discover ways to harness data in your business, simply send us a note today. Analytics Guild offers 1:1 coaching, group workshops, and advanced analytics services.