When AI Daydreams: The Business Implications of AI Hallucination

In the next articles, I will share with you several AI challenges that are keeping many AI practitioners awake at night and can affect your business. Let’s start with hallucination.

AI Hallucination

The use of AI in the industry is initiating a new revolution that will help us produce more, better, and cheaper. Industries will be completely transformed. But for humanity to handle the risks, we need first to understand the technology, not just experience it.

The AI became evident to the masses after OpenAI introduced ChatGPT and set a new record in user adoption. Two months after its launch, ChatGPT already had 100 million active users, becoming the fastest-adopted application in history. Threatened by its dominance in search engines, Google entered the competition by launching Bard AI Chatbot in February 2023.

The honeymoon was very short. Users realized that the models were far from perfect. The users experienced absurd, misleading, and offensive responses during the interaction dispelling the illusion of perfection. On the other hand, companies that rushed to integrate these tools realized they were exposing their reputation by underestimating the complexity of problems in Large Language Models (LLM) like AI model hallucinations.

What is AI Model Hallucination?

Hallucination, in the context of chatbot systems (Natural Language Generator - NLG), is the phenomenon in which a chatbot formulates incorrect or misleading responses presented with plausible and very convincing arguments. And worse, the system "believes" in what it is saying.

What are the Risks?

Depending on the business and where you apply the AI LLM, the company could face a variety of damage to their business.?Let's point out the three interconnected areas and players.

Areas

1. Reputation

Reputation is considered one of the most important company assets and is the first on the line. Incorrect or misleading information provided to the customer can cause irreparable credibility loss. WOM (word-of-mouth marketing) is exponentially amplified in social media and very difficult to revert it.

2. Legal

The customer perception of a company’s reputation could be surpassed by legal risks that are more tangible. Unfavorable court verdicts in regulated industries like finance and healthcare could be leading to operational restrictions, contract suspensions, and reparations.

3. Financial

Apart from the financial risks from legal issues, inaccurate data or misleading recommendations due to hallucination can lead to poor business decisions, ultimately causing financial losses. AI automation that involves complex business environments may introduce imperceptible distortions into the business processes, resulting in financial impairments.

Players

First and foremost, the biggest risk is for the creators of chatbot models. Google, in its first Bart demo, made a factual error in response to this question: "What new discoveries from the James Webb Space Telescope (JWST) can I tell my 9-year-old about?" Bart wrongfully stated that JWST took the first picture of exoplanets. The carefully chosen question to highlight its application in education failed to give a correct response. The market reacted by reducing the market value of Alphabet Inc (GOOGL.O) by $100 billion. Other impactful cases can be cited, such as Microsoft's Tay Chatbot, which was suspended after 16 hours of operation and 96,000 posts on X (formerly Twitter) due to disastrous interactions with users, and Amazon's AI recruiting tool accused of gender bias, as well as Google Photos misclassifying humans as animals.

Secondly, there is an emerging adoption of generative models by companies in various sectors. A frenzy that led to decisions perhaps without a complete understanding of the technology, its stage of development, challenges, and risks. So, what could go wrong? Everything. Who could have imagined that ChatGPT would set legal precedents when questioned by lawyers about legal cases to be used in legal briefs? As a result, a U.S. judge imposed sanctions on the lawyers responsible for the legal briefs. The law firm and its lawyers suffered serious damage to their credibility.

And finally, model hallucination also victimizes the technical community. The use of the tool as prompt engineering quickly gained traction in creating computer programs. However, problems like the suggestion of imaginary functionalities and inexisting applications are common and can result in security vulnerabilities and poor performance.

How to mitigate those challenges?

First things first - We need to educate ourselves to understand the technology, but far from the press releases. It is not just something for the engineers, but for the entire company to be educated. Then, we can critically think about the risks and mitigate them.

AI is not just another technology. It is a new paradigm that is deeply revolutionizing the way we live, work, and engage with the world and people. Educate yourself.

- Click here for references

When AI Daydreams: The Business Implications of AI Hallucination ? 2023 by Roberto Williams C. B. Batista is licensed under Attribution-NonCommercial-NoDerivatives 4.0 International

https://chooser-beta.creativecommons.org/