The ExpertData Dilemma

With world’s incremental dependency on Data driven, predictive decision making, we should be careful of managing the risk of not well integrating the human experience. I’ll start with a illustration by a joke:

Fall was upon a remote reservation when the Native Red Indian tribe asked their new Chief what the coming winter was going to be like. The modern day Chief had never been taught the secrets of the ancients. When he looked at the sky he couldn't tell what the winter was going to be like.

Better safe than sorry, he said to himself and told his tribe that the winter was indeed expected to be cold and that the members of the village should stock up on firewood to be prepared.

After several days, our modern Chief got an idea. He went to the phone booth, called the National Weather Service and asked, "Is the coming winter going to be cold?"

"It looks like this winter is going to be quite cold," the meteorologist at the weather service responded.

So the Chief went back to his people and told them to collect even more firewood in order to be prepared. A week later he called the National Weather Service again. "Does it still look like it is going to be a very cold winter?"

"Yes," the man at National Weather Service again replied, "It's going to be a very cold winter."

The Chief again went back to his people and ordered them to collect every scrap of firewood they could find. Two weeks later the Chief called the National Weather Service again. "Are you absolutely sure that the winter is going to be very cold?"

"Absolutely," the man replied. "It's looking more and more like it is going to be one of the coldest winters ever."

"How can you be so sure?" the Chief asked.

The weatherman replied, "The Red Indians are collecting firewood like crazy."

Earlier an expert who could predict weather changes by studying subtle hues in atmosphere, trusting senses, is now relying on the modern day experts which are more and more turning out to be data driven and the data they choose here seems to have landed in a loop.

Are we not in the same loop today ? We are asking experts for some advice and expert advice is based on the prediction, which is  based on some algorithms that have been trained on a data, which gets enriched when more and more data supports that algorithm due to expert advice.

More we are moving toward the data driven, predictive world, more humans are doubting their ability to judge and trust their instincts. Every decision today must be backed up by a study on vast majority of data , run though a model and presented visually. Machine learning  has broken down expert pattern of working which is based on experience, reflection, more experience and iteration. This has been done with so much expertise(pun intended) that many expert’s previous certainty has been discredited by new analysis. We can only hope that all the analysis has been done on valid, unbiased dataset. Because what machine can still not gauge is the intent and predisposition which an expert can.

 So what do we do now ? Trust an expert or analysis ?

 May be ‘Trust but verify’, moreover trust your instincts, that is a very complex trained algorithm.