The Wisdom of Crowds & Data Science

I wish to write this two years ago, it was in my "what to do" notes. I read James Surowiecki book at that time and got inspired.

The wisdom of crowds theory is simply about the idea that large groups of people will be more smarter than individual experts when it comes to predicting.

We could say that the roots of the wisdom of crowds theory originally thought by Aristotle (384-322 BCE). Moreover in 2018, Chao Yu et all did an excellent job summarizing the “crowd science” as a literature review. It turns out that the wisdom of crowd concept can be applied widely to problem solving, decision making, innovating and predicting, and in many fields such as finance, psychology, culture, biology, behavioral economics, engineering, marketing, and other fields. Interesting right?

In one of Netflix scene, the wisdom of crowds visualized as how accurate thousands of people are when they are guessing the amount of candies in a jar. The number of times their average collective guesstimates are accurate is so remarkable. 

In Yu (2018) research, some scientists believe that the value of group wisdom is closely related to the surrounding environment (Katsikopoulos and King, 2010). In a strange environment, the individual tends to be more agile, and the wisdom of the group will arise too. But bear in mind, crowds aren’t always wise. In fact, some can be the opposite.

Moreover, Surowiecki (2005) advised that the wise crowds should have 4 key characteristics as follow:

1. The crowd should be able to produce diversity of opinions.
2. Person’s opinion should remain independent and most importantly should not be influenced by others
3. The participant in the crowd should be able to make their own opinion based on their individual knowledge.
4. The crowd should be able to aggregate the individual opinions into one collective agreement. 

Why is it important to have wisdom?

I personally think that there is so much merit to the idea of wisdom of crowds. As petroleum engineer, my day to day work often dealing with number, data and information including interaction with other people or groups to deliver best engineering solutions. Processing the data to a decision making is not a straight forward process. Recently due to high demand on the data science; in January 2019 Society of Petroleum Engineers (SPE) launched Data Science and Digital Engineering (DSDE) focus group. Please join and subscribe to receive SPE DSDE monthly newsletter here: https://www.spe.org/en/dsde/about-dsde/

I believe someday, soon, with an assistance of new technology such as data science (data mining), IoT, AI, big data, and cloud computing, we will have an ability to reach collective intelligence and explore our best potential to solve our human complex problems. So don’t be afraid to be different and making mistake, we need diverse, open minded, collaborative work and independent to reach a wisdom in our day to day activities.

Meanwhile, data science define as a multi-discipline approach by using a scientific methods, processes, algorithms and systems to extract information, knowledge, and insights from data in various forms. So what is the relationship between wisdom and data science? In my opinion, the relationship is very close. Data can only be changed into information then turned into knowledge and become useful in decision making, if we value data wisdom.

Wisdom means the ability to use our knowledge and experience to make the best decisions and judgments. Is wisdom something that money can buy? Please feel free to share your opinion about it.

References:

1. Chao Yu, Yueting Chai, Yi Liu, (2018) "Literature review on collective intelligence: a crowd science perspective", International Journal of Crowd Science, Vol. 2 Issue: 1, pp.64-73, https://doi.org/10.1108/IJCS-08-2017-0013
2. Society of Petroleum Engineers, (2019) Data Science and Digital Engineering newsletter. 
3. Surowiecki, J. (2005), The Wisdom of Crowds, Anchor.