Take the "Big" out of Big Data

The term "Big Data" is swung out there in the world as this amalgamous beast, with very little defined parameters and lots of very brainy, geeky folks positioning for the guru or alpha geek title. Then there's the army of consultants and sales folks ready to propose the multi-million project that promises to cure all ails. But how do you bite off big data in little bites?

First let's level set, "big data" refers to the ability to analyze large sets of data in any format to look for insights, make predictions, and aid in decision making. Some folks fear the robot overlords where data and AI remove the human intervention, where mindless bots statistically cull the population and workforce. Ok, I can see that, knowledge is power, power in the wrong hands... ok not entirely unfounded, but you can use this power for good too :).

I've whole heartedly embraced Ginni's term "Augmented Intelligence" - where you leverage all the possible available data to make a decision, more data than you, the human, can ingest - and yes, sometimes domain knowledge, experience, and the real world means you dismiss the statistics and analytics and go with your gut. Human connection, emotion, motivation, hope - those are factors unlikely to be replicated anytime soon.

But why is there so much hype *now*? Well two areas have changed that has pushed this space to the dot.com era level of frenzy. First is the proliferation of data, so much of civilization and interactions are now captured electronically, from the activity in your appliances and what your Aunt Ethel had for lunch to sensors in deep space, weather, video surveillance and digital breadcrumbs. Not only has technology allowed for the easy capture of this information (look at how much is documented by smart phone, Snap Chat culture), but the ability to store and use large scale computing services is accessible to anyone with an internet connection and credit card. So the expense barriers of capture, store, and compute are virtually gone.

The second is the ability to analyze unstructured data. Long term established players such as SAS and SPSS have been able to do predictive analytics on structured data for quite some time. Columns and rows of data, specified data elements structured for automation. However, now we can access unstructured data- documents, images, video, so all that extensive data mapping/structuring negated by emerging tech and advanced algorithms. I've mentioned previously, there is almost a step back as far as hand coding and the holy war of languages. That noise will shake off over the next few years, funding will end for some, others will get lost in IPO fever, players will emerge and become established.

However the final step - the decision making - this is where you'll get your return. Actionable steps that will save money, lives, create revenue, opportunity. This is where IBM invested billions and over a decade in research and development, and to be honest, they're light years ahead. Seriously. Perfecting statistically rankings and algorithms that adjust with every new data point. Finding patterns and learning what's wrong and what can be trusted. However, that seems like crazy big projects and multimillion dollar investments. Ok, well GBS will be happy to create those career pivotal marquee engagements (hopefully good pivots;)) for you. But for those with leaner budgets you don't have to take that big bite - you can baby step into it. And that's where we/I can help.

Take pragmatic steps. Each small step should inch you forward to those seemingly science fiction solutions. And with each step - deliver something of value, so your organization will be willing to invest again. You can definitely jump all in with an engagement, but if your organization has to wait 6 mos, 12 mos to get to something to aid in decisions - they'll lose patience and you'll lose credibility. And too much tech, especially in older organizations can just scare the bejesus out of your users, they won't trust it, and they won't adopt it, then they'll resist, undermine, or go around it. So the key is tasty little nibblets of valuable insights along the path. Deliver, measure, and adjust.

You can also begin with basic repositories, automated discovery, and start layering in more advanced models and functionality. Most of this tech is available in API's, some are completely free - much to the dismay of some of software resellers. And leveraging Watson Data Platform, built on Spark, to connect all your data sources, people resources, tools, API's and Applications - well, that negates the future rip and replace or migration barriers to delivering value.

So pick your glaring use case - workforce planning, marketing, supply chain, purchasing - wherever you're bleeding - whether that's hard costs or lost opportunities. What do you want to do, what information do you need to make a decision, what do you have on hand, what are you missing? Let's fill in the gaps and get something delivered to make those decisions - not in 6 months, but in as little as 10 business days. Or heck, with Watson Analytics you can be looking at insights in minutes.

Pick my brain, I've had to pull together data for executive decisions with no budget and no time and no resources and paychecks literally on the line. So I'm happy to call BS on the onerous requirements someone's put in front of you. I'm happy to share the world of possibilities and brainstorm. It's my passion and I'm more than happy to help.

If you need help with licensing, resources for deployment, training, user adoption - well I have a whole team right here ready to invest in your journey and your success. Here's a little about my happy new family -

Why Aviana?

Our story: https://vimeo.com/106965002

Aviana has documented over $1.8 billion in ROI for their clients. The overwhelming majority of Aviana clients are repeat clients because we keep working until our client is successful. We give our best advice and service regardless of profit.

Lillian Taylor, Aviana Global IBM Watson & Emerging Technologies Adviser

Incurable Entrepreneur, Sherpa to IBM, Watson-BigData-IoT-Blockchain Evangelist, Business & Channel Development, Analyst