Big Idea 2015: Big Decisions Will Supplant Big Data

In this series of posts, Influencers and members predict the ideas and trends that will shape 2015. Read all the stories here and write your own (please include the hashtag #BigIdeas2015 in the body of your post).

I predict that 2015 will be the year when big decisions supplants Big Data as the key initiative in business, government, and society. We have all been focused on the idea of Big Data for a few years now. We have more sensors collecting substantially more data about more people, places, and things. And as computing power becomes cheaper, there is an almost limitless collection of data. And the data is dissected, analyzed, and admired in a numbing parade of charts and graphs.

In some leading cases, the data has helped make better incremental decisions. Decisions like “what am I really searching for?” Or “where is the best table available for dinner on Thursday?” Or “what else would this customer like to buy from us?” Some bigger decisions, like where to point cancer research, how to get humans to Mars, or how to evacuate hundreds of thousands of Filipinos in the path of a typhoon, are starting to be better informed by Big Data.

That said, many leaders know their company or government could move much faster and more boldly when it comes to converting data and analysis into decisions and action. Unfortunately, the rise of Big Data has not eliminated “analysis paralysis.” In fact, in some ways it has actually even enabled indecisiveness. Because as more data has become available, the easier it has become to get lost in details, and the more tempting it has become to want the data to make the decisions for us.

Our ability to turn Big Data into “big decisions” is more critical than ever. That’s because we have only begun to realize our full potential when it comes to using Big Data to prompt the big decisions that will help us solve the most daunting set of problems our species has ever faced. Those problems include the massive, highly complicated issues like managing climate change, overcoming scarcity water and food, and providing accessible health care for more than 7 billion people. They also include smaller, but frustrating, issues like the unnecessary loss of human life because of simple things like texting-while-driving or the lack of flu immunizations.

With each of those problems, we collectively have an enormous amount of data illuminating the causes, effects, implications and potential solutions. And yet, because of the huge scale and complexity, we collectively fail to take sufficiently decisive action.

How do we move from being excited about Big Data to realizing data is just a means to the ends of big decisions?

In their book Analytics at Work, Thomas Harris, Jeanne Davenport and Robert Morrison point to leadership as the single most important factor in a company’s ability to turn data into good decisions. They go one to say that: “The best decision-makers will be those who combine the science of quantitative analysis with the art of sound reasoning." I agree, and predict that 2015 will be a year when we move from Big Data to Big Decisions by accelerating sound reasoning to match the growth in quantitative analysis. These changes will happen on three levels:

1. Big individual decisions. We have more individual data than every before. Data about how we are driving through in-car sensors. Data about our health from smart watches and other wearable computers. Data about our cognitive abilities from daily tests on our smartphones. Data about our finances from multiple apps. Data about the traffic patterns on our commute. In 2015, more of these data sources will add reasoning to help us make decisions and take action that improve our health, improve our safety, or improve our financial choices. We are even evolving the weather forecast to become more personalized to your life outdoors -- with forecasts for traveling, boating, surfing, running, meal planning, hair care, and 100s of other use cases. We have learned that turning data into decisions helps people make the most out of weather data, and I am sure other data analyzers and forecasters will do the same.

2. Big company decisions. Companies have been investing heavily in Big Data. They are tracking every SKU, every purchase, every ad impression, every referral, every hotel stay, product efficacy, and so on. Companies have been correlating the behavior of their customers with economic data, pricing data, weather data, news cycles, and every potential factor that might drive a customer decision. The Powerpoint decks have gotten thicker, but have the decisions really gotten better? In a few leading cases, companies are getting more savvy about using the data to drive better and more precise decisions. We see better decisions to serve the right content to each customer from savvy recommendation engines of leading e-commerce and publishing companies. We see better decisions from retail ad campaigns, synced with sophisticated supply chain management, which vary by geography based on local weather and economic conditions. We see better investment and risk decisions in some technical trading models. But most companies are not yet using machine learning, not are they automated decision-making with better algorithms. Instead, they are feeding more data into the same human processes. The decisions are constrained by the human capacity to process the data, so only a small part of the Big Data is actually used. In 2015, we should see more companies rethink their decision processes to make Bigger and Better Decisions using Big Data.

3. Big societal decisions. Government officials and large non-profit leaders need to make decisions that affect society as a whole, in policy areas ranging from sustainable energy to education and healthcare access to deployment of national and international security efforts. To help make these decisions, governments have also long been the biggest collectors of data -- satellite observations of the Earth. geological surveys of the planer, economic and employment data, census data, transportation and traffic patterns, and millions of other topics. This data is compiled, released in summary reports (often weeks and months later), and used to help lawmakers and policymakers with Big Decisions. But recently, there have been efforts in the USA and other key governments to make the data accessible more rapidly and in more raw and granular fashion. And this allows the data to be used by tax-paying citizens and businesses to improve their own decisions. It also helps create entire new business models, like our weather business, or Google and Microsoft's map business, or Bloomberg's economic indices. When we free Big Data, process it through faster and lower cost computers, and turn it into Big Decisions, we create an economic engine with more precision, more efficiency, and more time left for innovation,

In the past few years at The Weather Company, we’ve talked a great deal about the importance of becoming a distinctively great “data company.” In 2015, we will move toward becoming a great “decisions company,” a company that provides our customers not only with good information, but also doing so in a way that helps enable the people we serve to make good, timely decisions based on short and long term weather predictions.

That commitment has required us to become better decision-makers ourselves, and it’s a capability that must become a more prevalent part of who we are and how we operate.

As we strive to do that, I’m curious to hear what you are seeing your world? How well are you turning Big Data into the Big Decisions that create the Big Ideas that make the world a better place?