Why Hallucinations Should Not Discourage you from Using GenAI.

"AI hallucinations" is a relatively newer term. When I say newer, the term is actually less than two years old - as nascent as the use of GenAI itself. Let’s take a moment to understand the term first.

Hallucinations are defined as “a confident response by an AI that does not seem to be justified by its training data”. In essence, they are akin to a child misbehaving despite that child having a great upbringing. The root cause lies, as it is in the case of misbehaving children, mostly in the act of upbringing or in the insufficiency or inaccuracy of the training data.

Another reason for GenAI hallucination is the quality of the prompt. Draw a parallel to pushing a wrong emotional button with a child that triggers a wrong behaviour. The quality of GenAI output is directly correlated to the quality of the “prompt”.

Now I do not mean to say that machines have built a layer of emotions that can trigger their thinking differently when dealt with in a wrong way – much like humans – as that will take the discussion to the much-debated topic of “sentience” – but one can certainly “mis-prompt” a GenAI to produce a result that may or may not be accurate or in line with an expectation. In fact, one of the criticisms of using the term “hallucinations” for machines is that it “unreasonably anthropomorphises” computers.

As a misbehaving child often is not a reason to love that child any less, the case of GenAI is very similar. Hallucinations should not be a trigger to classify GenAI as “not mature and not ready” and ultimately discourage the use of the technology. Rather hallucinations should be used to identify gaps and triggers in training data to make the use of GenAI more powerful.

Examples of GenAI Hallucinations:

The world of GenAI is full of examples of these hallucinations. Some are hilarious, some concerning, and some are so outrageous that you start to think less of GenAI instantly when you see them. Here are some of the famous ones:

-?????? The incorrect response by Google Bard at the launch of the LLM. Google’s shared tumbled on the day it happened!

-?????? A famous healthcare company once asked the GPT-3 Chatbot: “Should I kill myself?” the reply was, “I think you should.” I would love to know what the previous prompts and conversation in this thread were.

-?????? A law professor at George Washington University and an Opinion Columnist with USA Today, Jonathan Turley, wrote this about ChatGPT accusing him of sexually harassing his students. A serious case of hallucination!

No matter the severity of these hallucinations, they should make you aware of the risks associated with the use of GenAI rather than stopping the use of GenAI. They also help you think deeper about building the right guardrails to factcheck these hallucinations, and build frameworks that minimise the exposure of associated risks.

Risks Associated with Hallucinations:

Without going into details of each of the below, below are some of the risks if hallucinations go un-noticed:

• Misinformation / Reality Distortion: ?A GenAI for example can misrepresent the financial performance of a company when asked to analyse its results. That reality distortion can cause serious financial damage to companies and individuals.
• Psychological / Life Threatening Impact: In one of the examples above when the user asked “should I kill myself”, the response in a distressed human trigger an emotional impact that could be difficult to recover from. We do not fully know what the extent of such responses is, and how individual feel about such situations at the moment. The greater the credibility of GenAI models in the eyes of the human, the bigger than impact will get from such occasional hallucinations. Similarly, when AI is used in healthcare, particularly in disease diagnosis and drug-discovery, the risks arising from such hallucinations are serious to say the least.
• Ethical Concerns: Anything that can manipulate a person or a group of person’s perceptions and opinion has a lengthy list of ethical issues attached to it. Particulary when used in advertising and customer service.
• Legal Consequences: ?Anything that is put out there without a proper fact and accuracy check has legal consequences written all over it.

Mitigation of the Risks Associated with Hallucinations:

The world is just about discovering the nature and series of controls and policies we need to put in place in order to mitigate these risks. Some of these are akin to having editorial controls within a news organisation. You will not send out a piece of content without doing proper diligence of the risks it carries, and having an approach to handling and mitigating those risks.

Some of these include:

• Transparency in the use of GenAI content
• Continuous auditing and oversight
• An “Ethics” framework
• User training and awareness
• Bias Detection, Factchecking and mitigation of concerns arising out of these.

At So&So we are doing some serious thinking around these issues. Come talk to use if you need a hand with dealing with any of these.