Your Thirst for Information is Killing Innovation

In 1974, world renowned psychologist Paul Slovic ran an experiment to evaluate the effect of information on decision making. He gathered eight professional horse handicappers – people who are considered experts in predicting winners in their field of horse racing. His challenge to them was twofold:

1. Pick the winners

Predict the winners of 40 horse races over the course of four rounds, with each round giving them access to an increasing amount of information. In the first round each handicapper could ask for any 5 data points they desired: for example past win/loss results for the horse, years of experience for the jockey, etc. In the second round they’d then receive 10 data points, third round 20 data points, and in the fourth and final round they were allowed 40 data points

2. Predict your own win rate

With an average of 10 horses in each round, the probability of picking a winner at random without any data points would be roughly a 1 in 10 chance, meaning 10% would be a sensible confidence level. Slovic asked the handicappers to also state their level of confidence in picking a winner in each round as they gained access to increasing amounts of information.

So how did they do?

In round 1 with 5 pieces of information, the handicappers on average were 17% correct in picking the winner – a great result, meaning a 70% increase in accuracy when increasing the amount of information from 0 to 5 data points. On top of that, their confidence level had been 19% on average, almost an exact prediction of how many winners they’d pick.

Unfortunately in the next round with 10 data points, their win prediction remained at 17% – no improvement. This trend continued through the remaining rounds; even with 40 data points, their win rate remained at 17%. The handicappers were literally no better off with more information. Even worse, their level of confidence in their predictions had swelled to 31%, almost double their actual win ratio. The additional data points gave them a false confidence in their ability to predict winners, drastically increasing the likelihood of making large bets that actually turn out as losers.

The lesson: More information doesn’t improve the ability to make good predictions, and can actually create a confirmation bias that leads to bad decisions.

Investing in Innovation: No horsing around

In order to succeed in the field of innovation, the skillset required looks less like the individual who is great at driving operational excellence, and actually has more in common with the traits of a successful investor. Innovation is not a game of incremental gains, but by nature achieving something that has not been done before. This often means playing in an area of unpredictability, where even the most complex models can fail to pick the correct trend or make an accurate prediction. Does anyone remember when former Microsoft CEO Steve Ballmer held a funeral for the iPhone, anointing Windows Phone as the iPhone killer?

In traditional business schools, investors are told and encouraged: the better informed you are, the better your investments will perform. It makes sense on the surface, but this has a lot to do with our conditioning in the business world – this need for information becomes counterproductive when the task at hand is to discover and capitalize on new opportunities.

Adam Robinson is a New York Times best-selling author and a former US Chess Federation Master. These days he advises some of the world’s largest hedge funds on macro investment strategy. He was recently interviewed on Tim Ferris’ Tribe of Mentors podcast where he recounted Paul Slovic’s horse handicapper story. His takeaway?

“Beyond a certain minimum amount, additional information only feeds confirmation bias. Leaving aside the considerable cost of acquiring that information, the information we gain that conflicts with our original assessment or conclusion we conveniently ignore or dismiss, while the information that confirms our original decision makes us increasingly certain that our conclusion was correct”

The Death of the Business Case

We see examples of confirmation bias all the time in the real world.

Traditional wisdom in the field of strategic planning is to “de-risk” all decisions by first creating an air-tight business case. This can often involve months of data gathering and number crunching, which in itself represents significant time and resource cost to gather and develop. Investing in anything in the absence of this level of “rigour” is seen as reckless and is rarely able to garner executive buy-in.

The process is tantalisingly simple: Define our current position, determine where we want to go, then figure out what needs to be done and create a three-year roadmap of tactical activity required to get there.

As-Is, To-Be, Gap Analysis, Roadmap. Playbook done, let’s go home.

The dirty little secret is that this process typically begins with a hypothesis – a first stab at the final answer. Approval is given to gather information to validate the initial point of view, but this investment is typically used to simply post-rationalise the original hypothesis. The project team would look silly if their first guess turned out to be wrong, right? Sound familiar?

Worse than simply leading to bad investment decisions, this “de-risking” process can in fact be fatal when it comes to innovation efforts. Robinson continues:

“Because Financial Trends involve human behavior and human beliefs on a global scale the most powerful trends won’t make sense until it becomes too late to profit from them. By the time investors formulate an understanding (again a false understanding) that gives them the feeling of confidence to invest, the investment opportunity has likely already passed”

The Do’s and Don’ts of Innovating

When the intent of innovation is to break new ground, does it really make sense to gather data on what others have done and wait for someone else to make the first move? Surely there must be a better way.

Here are five do’s and don’ts to keep in mind when planning an innovation initiative:

Don’t: Spend time and effort on a broad market sizing exploration in an attempt to please everyone. This information will only be broad and shallow and only serves to make stakeholders feel safe without actually reducing risk in a meaningful way.

Do: Go narrow and deep – solve a real problem for a single customer persona. It will be much more impactful and will allow opportunities to scale over time.

Don’t: Use a traditional business case to gain sponsorship for innovation initiatives. Trying to demonstrate ROI of the unknown is an exercise in futility. The information will be vague and won’t be compelling no matter how great the powerpoint deck.

Do: Gain executive buy-in by showing, not telling. Focus on building out a lean POC to demonstrate that there is a business opportunity. Even if it’s just a Business Model Canvas with some validated assumptions and a real customer problem, it will be more compelling to share some real untapped insights, not just observational data points from what’s been done in the past.

Don’t: Spend resources on trying to develop a lengthy three-year roadmap. The reality is the playing field will have changed 12 months in and the rest of the plan will need to be scrapped.

Do: Zoom Out then Zoom In. Think of what the broad vision might be 10+ years from now, then quickly define the immediate steps over the next 6 months in order to start moving in the right direction.

Don’t: Fall into the planning fallacy and get caught up trying to map out fixed phases and milestones. The information needed to execute on innovation doesn’t exist in a planning spreadsheet or in historical project data.

Do: Plan and execute in agile. Assume you won’t have all of the right answers, so get something to market quickly, fail rapidly and continuously adapt based on real world data.

Don’t: Just gather information that supports a pre-baked hypothesis. Assume half of the information you need is misleading, and the other half doesn’t exist.

Do: Use a customer-centric method. Map out a real journey to uncover all of the things that might deviate from the obvious business case. Go and talk to real customers to understand what’s really happening in their world.

The verdict is in – your favourite business school professor misled you, despite the best of intentions. When it comes to innovation, the amount of information you gather can have an inversely proportional effect on the quality of your decisions. Besides taking up considerable time and resources to gather, it actually feeds confirmation bias and makes us overly confident in incorrect hypotheses. The trick to innovating is to gather just enough information to make the first move, putting our emphasis on solving a problem for a segment of one and continuously adapting in a live environment. With a few simple tweaks, your organization’s innovation efforts could be off to the races – and that’s something you can bet on.

SOURCES:

• https://tribeofmentors.com/
• https://www.listennotes.com/e/6124734bf3bb47f3be10ba4b82d0f739/adam-robinson-the-power-of-the-unconscious-mind/
• https://techcrunch.com/2010/09/10/windows-phone-7-funeral/
• https://deloitte.wsj.com/cfo/2017/09/20/crafting-corporate-narratives-zoom-out-zoom-in/