Algorithms and Big Data are NOT substitutes for decision making

The aim of this article is to throw light on the other side of the coin. I am no way refuting or disparaging the importance and benefits of Algorithms and Big Data. In this article, the objective is to provoke discussion on the topic of risk aversion in managers and misplaced thinking that data is a panacea to all issues. Observations, comments and anecdotes are keeping in mind for profit companies in the Asian region.

Algorithms are heuristics to accomplish a task and because of machines’ computational power and prowess, they are able to deal unwieldy data efficiently than a human mind. However, an algorithm is in a way its author’s view of how things ought to be or solved or arrived at a solution and therefore it inherently carries a bias – most of the times without malice and unbeknownst to its author.

Nowadays it is fashionable to hear in meetings and discussions about how companies need to take advantage the big data they have and how it is leaving money on the table by not harnessing this trove. In most of these talks (some that I have been part of and others anecdotal from colleagues in other companies), many executives believe as if Big Data will reveal something like a gospel which will remove the issue/s at hand or turnaround to profits. Two key things are ignored in such discussions and these are in a way bedrock of analysis and output, these are:

·        Data Integrity

·        Insights

Let me elaborate: since historical data is used to analyse and then come up with patterns, segments etcetera; it is imperative that such historical data is not marred in any way. If it is, then such bad practices or issues will be reinforced and solidify in future. For example, if Fox News was to use its past data to shortlist or identify employees for future leadership roles and data for past 20 years is analysed; the output will be marred because of the way company worked under Roger Ailes. Results will most certainly will be disadvantageous and discriminating to women because the past data is impaired.

Another example is Multi Health Company – a US based company that uses algorithms to decide if a prisoner is worthy of a parole. Imagine the bad data it is dealing with, no wonder that its results reflect the bias. It is not the fault of the algorithm but in both the above examples, the data the algorithm will rely upon to arrive its results is bad. The algorithm will do a GIGO: Garbage In Garbage Out.

Similarly, if data is not overlaid with insights then it may not be beneficial because as of now our algorithms and AI are not sufficiently advanced to capture nuances. For example, few years ago Netflix crunched its viewership data and realised that most viewing is happening on weekends of different shows but similar genre. No surprise there but what they did with it is layer with an insight that viewers love binge watching in batches. Soon Netflix started releasing all episodes at one go. What was not revealed in the data is what viewer wanted. The person looking at it connected the dots and viola they captured many happy customers and increased subscriptions

Another example is how Amazon Video and Netflix looked at similar data and deduced that people preferred to watch a programme with senators as protagonists. Whilst Amazon’s team stuck to the tee on results and produced a show with four senators, (the data supported this result) and ended up making the TV show ‘Alfa House’. On the other hand, the team at Netflix overlaid the result with their insight and went ahead and made House of Cards with machinations of power and politics, the rest is history.

The point being that blindly following data may not be the best strategy. Of course, nobody can fire you for doing so because you can always lean on data to show that it guided your decision. However, it misses the point, managers in companies are required to overlay findings with insights and then take risk or leap of faith. Big Data in ways is giving easy way out to managers to absolve themselves of risk taking because a decision comes with accountability. Organisations need to foster an environment where mistakes are not penalised otherwise employees will play safe and we know such companies are on their way down sooner rather than later.

Data is a tool that should aid executives and managers in arriving at decisions. Things go wrong if algorithms and data alone start driving decisions whether in business or otherwise. Many executives mistake that anything that is hi-tech and involves complex algorithmic computation should be better or superior. It is a flawed reasoning because as mentioned earlier algorithms are heuristics and not absolute mathematical truths; that is the reason Google, and Bing give different answer to the same query.

To conclude I believe there are immense benefits of Big Data and Algorithms but it does not absolve managers from applying their experience and reasoning in arriving at decisions. Data is looking at past and we move forward looking at the windscreen but with only an eye on the rear-view mirror.

========================================================

Acknowledgments

I'm thankful and inspired by the many TED talks on this subject, various Articles and Discussions that helped me understand the topics. Examples cited in the above article are from such talks and articles. To books that I have read on the topic of big data, algorithms, innovation and leadership that helped me in penning this article.

?All images are from www.pixabay.com

? Mir Amer H, 2018. All Rights Reserved.