Misrepresented decisions: Cost of capricious data systems

Towards the early 90s air travel was made so efficient and safe that the late ruler of the Emirates HH Sheikh Zayed while addressing his citizens on road safety mentioned: "The road travel is much more dangerous than air travel". Statistically speaking he was true, as the odds of being on an airline flight which results in at least one fatality is 1 in 3.4 million. This was a win achieved through precision engineering, rigorous training of the pilots, stringent process policies, checklist, protocol ... The two names that supplied the aviation industry with aircrafts Boeing and Airbus, that ran into the cut-throat competition and often bet against the winds had by then earned the trust of the fliers as the safest to fly. Not every time the win was for the same company, while despite the engineering marvel of the A380, the business evolution favored the Dreamliner. When the A320 picked up the race on the single-aisle, the Boeing in no time responded with its 737 max, the most celebrated, heavily automated, technologically advanced solution for the aviation... until it cost 346 lives in two consecutive crashes in a span of 6 months.

While the instincts were to blame the pilot, weather and external factors, the dirty story of the malfunctioning of the Maneuvering Characteristics Augmentation System (MCAS) came to light, which puts the entire tech world to a scare. The loss of lives along with the resources, finally called for a grounding of the entire fleet across the world as we stand today.

One should note that this comes at a time where the industry bets on digital, data-driven, automated decision engines as never before. Both the flight failures now point to a faulty erratic decision due to dirty data sent from sensors, where the pilot could just remain a helpless witness to the fatal scenarios that spun off. Such is the cost of decisions driven by contaminated data.

While there were a lot of isolated differences against transferring the absolute autonomy of decision making to the data machines in mission-critical scenarios like that of a pilot, a driver, and even a CEO, they were soon pilloried by the automation wave, the digital transformation promises and overall by the hyped call for AI and ML. Today, every company irrespective of its business tries to wrap up their entire story with the help of data description, mostly knitted under pressure, competition and industrial FOMO. Know it or not, you have to speak of regression, plots, and trends to secure your funds or position. What is more alarming is that in the race to knit this story, the aftermath of the decision that the story would lead to is often overlooked or is unknown. While in short window times, insights backed by data that lacks validation shall woo the business world, not soon would one see it a bust. We do have enough examples across sectors, be it the mortgage crisis, the faulty weather predictions, the upturn of legacy firms or even the failure of political dynasties.

Very often the one who calls out the data bluff would be termed as a conservative, or a roadblock in development. Even the pilots who called out the menace of over automation in the aircrafts were called such, as they bet against the least probable, until one day it did turn out a mayday twice.

Data has always transformed the way we do business, the way see and understand, the way we communicate and overall our entire lives. As data professionals and data storytellers, it is indeed their responsibility to ensure that the narrative they put forward is backed by sound and solid data, that have an address and have a pristine degree of sanity. We as humanity have agreed on fake news as a menace to that hampers and destroyed nearly anything the race as humans have achieved till today.

What better are misrepresented insights from dirty and contaminated data than fake news?

Data systems and solutions are the guidelines and navigators for any operation: business or engineering solutions. They are to designed against failure and not designed to fail. Be it machines, precision instruments, businesses, what data narrative provides is magical and beyond the capabilities of human intuition, but the very same systems shall be the one which pulls the plug. Most institutions that run with the 80-20 approach tend to call the unexplored systemic errors as random errors. One can simply imagine the implication of the approach in scenarios like that of prediction, budget planning, and estimations.

We know today even with the perfect model, we already are short of infinite precision to predict the events of the future, then what do we have when our data sources of the model itself are capricious?

The countries and bankers that once thrived on the oil of the deserts are today busy investing in the data-savvy companies: they call it the new oil. The trust in oil by its consumers was earned by the arduous efforts to create pristine products through fractional distillation, had not that been successful the oil would have remained black oil and not black gold. As we see the data hype is flattening and the trust in intuition seems to be taking over the trust on data. It is up to us to ensure we provide pristine and pure insights out of the gold mine we have, else we shall soon bound to be trashed for the good of our economy and the human race.