Data DATA everywhere - not a drop to drink..

I had some personal experience in 1990 with the ancient mariners' rhyme or The Rime of the Ancyent Marinere:

Day after day, day after day,
We stuck, nor breath nor motion;
As idle as a painted ship
Upon a painted ocean.

Water, water, every where,
And all the boards did shrink;
Water, water, every where,
Nor any drop to drink.

I was the newly appointed Chief Mate on a ship with an intransigent bunch of crew from a very aggressive Marxist union. I will never work out where it was a sabotage (to secure a fairly high hardship allowance and compensation), or an accident - but somehow sea water gained entry into every fresh water tank on the ship (but that is a long story about unions and compensations).

The captain (and I) was left with a dilemma, whether to rely on the fresh water generator on board for a long sea voyage, or not. There was no shortage of water - it just was too salty for most purposes. I will tell you how this event panned out at the end of this blog post. But I want to change course here to the real purpose of the post - DATA, DATA everywhere. In my last blog post I recounted a real life story of a business transformation project where information technology fell woefully short.  This is not the only such situation I encountered. In fact I recount more than 20similar examples in my book 'UNCHAIN YOUR CORPORATION'.

 Look at the growth in global data storage capacity below:

By Myworkforwiki - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=29452425

Obviously it is not just the capacity - but also the data that is growing:

There are many other graphics showing similar growth of data - here, here, here and all over the cyberspace. The key story they all are telling is just one - WE ARE FLOATING IN DATA. 

Just like the ancient mariner had no dearth of water, we have no dearth of data. And, just as minute quantities of salt (3.5%) that is present in all that seawater, is enough to make raw seawater unusable for most practical purposes, minute quantities of data error is enough to make most of the raw data unusable for practical purposes. 

On the ships (and in many locations on land) we deploy fresh water desalination plants such as these. The sole purpose of these massive plants is to parse seawater, take it into a pressure chamber, evaporate it at high temperature (because that requires less energy) and then condensate it into distilled water. Further processing is required to remove other contaminants such as bio-hazards and others. Here is the full process in some more detail. 

The biggest opportunity in information technology is not to generate (or collect) more data, but to make the existing data more usable. I was asked a question at a recent speech why I was not as bullish as everyone else on big data. I likened the current big data set-ups to an ocean full of seawater. It still takes a huge expense to desalinate the seawater, and to make the data usable. 

Any start-ups that figure out a better way to collate, parse, access, and make usable the data to create insights would be a tremendous success. If you know of any, please let me know in the comments below. 

In the story above, we had to put into an emergency port to get fresh water rations without which none of our sailors would have survived. In the emergency port, we had a fight with the sea pirates (which is a story recounted in a previous blog).