Machine-Bias & Machine-Complacency: A Look at the Buzz around Generative AI through my Automotive Lens

Hype & Exuberance : The new season

We have lived through the media hype of autonomous technology, over the last decade, that made us believe that fully robotic transportation is just round the corner and will soon take us anywhere, anytime at our beck and call. There was even talk of a million robotaxis by 2020 that could eliminate taxis or other public transportation. Most now recognize that it is a much harder problem. While technical progress continues, the hype and exuberance have mostly dissipated.

If you follow news, even moderately, you might notice that the exuberance seems to have shifted to AI, Chatbots in general and one software, such as ChatGPT or any of its peers. My Google query for the word “ChatGPT” and the question “What is ChatGPT?” returned 883M and 568M hits respectively. Half of my news feed lately is about how AI is here to change the world. It’s a story about progress of technology. But it is equally a story of today’s buzz-centric media, hype cycle, viral spread and ubiquity of opinion channels, unfounded or otherwise.?Media and people are starting to project onto these tools (e.g., ChatGPT, Stable Diffusion etc.) qualities, capabilities and superhuman characteristics that it may not have, at least not yet. It’s like a Déjà vu if you remember the noise around autonomous transportation just a few short years back.

GENERATIVE AI

Generative AI, of which ChatGPT is one instance, has justifiably hit the sweet spot of our collective fancy. A whole cottage industry has mushroomed giving guidance on how to use the best keywords or questions to generate articles or letters or create apps for you. While homework essays or writing codes are some of the two most talked-about applications, people are also asking emotional question, personal problems or other complex decisions or issues not distillable from patterns of terabytes of words stored in the world wide web. There lies the crux of this article.?

MACHINE-BIAS & MACHINE-COMPLACENCY

When it comes to our relation to machine, especially those deemed as ‘intelligent’, we are prone to be influenced by two related sentiments – one we can call ‘Machine-Bias’ and another ‘Machine-Complacency’[1]. We have seen this at play in early years of autonomous vehicle development, when media and YouTube were filled with examples of “fully autonomous cars” making un-supervised moves and with predictions of an impending new world free of drivers. While the anticipation of the journey to the future was justified, the hype and the predictions were peppered with the so-called “Automation-bias” and “Automation-complacency”.

The first one, ‘Automation-bias’ refers to our human tendency to accept imperfect actions from an automated or machine system, even in the face of contradictory evidence. In our exuberance and astonishment about some of the human-like maneuver of a partially automated system, we often overlooked some of the errors or gaps in performance that would be unacceptable from a human operator. We can see similar bias in luxury ship liner Titanic’s failure a century back. While there were many factors contributing to the failure (including metallurgical failure of the hull that did not yet mature to hold its integrity in icy water), there is some anecdotal evidence (that we cannot fully verify) to suggest that officers on board the Titanic might have been overly confident in the ship's ability to navigate through icy waters and ignored some obvious warning about the dangers of floating icebergs ahead. If true, that is an early example of machine-bias or automation-bias!

The second one, “Automation-complacency” refers to our “over trusting” a half-baked automated system in thinking it will do the job in all situations, even in situations that it was not designed for. The best examples are many of YouTube videos of drivers boasting how they were able to take a nap when the car, designed only to keep itself on lane, was able to march on! Unfortunately, there were few who were not alive to tell about this complacent behavior.

BACK TO ChatGPT

As we will see, ChatGPT is still a bit away from the Asimovian robots that can be our omniscient companions or original thinkers that provide authentic answers in addition to taking over all mundane chores.

According to AI experts that I talked to, the Large Language Model (LLM) that ChatGPT and its peers are based on relies on existing text stored in the web. It primarily uses a conditional distribution of the next word given the previous to generate a statistically plausible contextual output. When asked a question about a topic in science, ChatGPT will likely generate a reasonable response based on a likely sequence of relevant words from the repository of scientific papers on the subject. That does not mean it knows science. ?Nor does it guarantee that it is scientifically right.

ChatGPT or similar tools may be designed to process complex language patterns and generate a “plausible” or probabilistic response. However, society and media are quick to attribute far greater super-anthropomorphic qualities to the software.?

LET'S TRY SOME QUERIES

I wanted to form my own relationship with ChatGPT. Last week some of my friends reported on or posted beautiful photos of northern lights right in our backyard in northern Michigan. So, I asked ChatGPT to give a poetic description of the northern lights. I got this:

“A dance of light in the night sky, The Northern Lights, oh how they fly,

A show of colors, a mystical sight, A heavenly wonder, a true delight”

That indeed was awesome, beautiful and seductive!!?

Then I tried something a bit trickier. I asked if there is proof of the famous Pythagoras Theorem using only trigonometry. ChatGPT said ‘Yes’ and gave a very plausible but faux explanation, using a circular argument. For those interested in the math rationale for rejecting this circular argument, see the box below.?

LARGE LANGUAGE MODELS (LLM) & TRUTH

Quoting Subbarao Kambhampati[2] (or simply Rao as we know him), renowned computer scientist and the past president of the Association for the Advancement of Artificial Intelligence, Large Language Models or LLMs “learn the patterns and correlations between words and phrases — the information that is stored in the billions of weights defining their trained networks. What this means is that when ChatGPT is “completing” a prompt, it is not doing this by copy-pasting documents but, rather, by constructing a plausible completion on the fly, using its network.”

However, its contextually plausible answer can not or does not know the reality or the innate truth or falsity of the topic, but “it is simply afactual” And “ChatGPT can give a highly relevant-sounding answer to any query, whether it involves grade-school essays, text summarization requests or questions involving reasoning and planning — but there are no guarantees about the accuracy of its answer”

?And to be fair, the landing page of the Open AI does provide the warning.?

THE TAKEAWAY: Déjà Vu of our Machine Bias/Complacency

LLMs (large language models) look at everything we say as a textual prompt to be completed with a reasonable set of words, that is plausibly coherent and potentially right sometimes, but without any guarantee to be actually correct, whether you’re looking for a panacea to your depression or wanting to write the best love-letter, a recipe, a resume or a piece of code or a question about contemporary science. The big progress in AI and technologies such as LLM notwithstanding, we are still humans and our age-old tribal tendency for machine-bias and machine-complacency seems to be too quick to embrace the super-human magic of LLM based AI software, overlooking the warnings of its potential inaccuracy, as we are increasingly inundated by the ubiquity of media buzz, viral sharing and un-constrained opinions.

Welcome to the Generative AI! Just like the development of autonomous technology, it is going to be an interesting and complex journey towards progress.

Disclaimer:?The comments, thoughts and opinions described are purely author's own and do not represent the position of either his employer or anyone else.

[1] The Glass Cage: How Our Computers Are Changing Us, Nicholas Carr, ISBN: 978-0-393-35163-7

[2] https://thehill.com/opinion/technology/3861182-beauty-lies-chatgpt-welcome-to-the-post-truth-world/