Snacking on Strawberries with ChatGPT

….Helping make the LLM worlds a better place, maybe!

What does your typical, friendly, tech sales guy do over the weekend, while waiting in the parking lot running family errands…??

They call on ChatGPT!?

On one such occasion I was fiddling around with ChatGPT and decided to test a common folklore about LLM’s - viz. the following -> Apparently they cannot correctly count the number of ‘r’s in ‘strawberry’.

(Sidebar - I also decided to check if this experience could be monetized…! ??)

It turned out to be a fascinating conversation!!

The below diagram summarizes the conversation. The subsequent notes outline what happened next. Lastly the important highlights…some hilarious, some stubbornly hallucinogenic and some really insightful.

What Happened Next…

• I sent the email to OpenAI highlighting the issue.
• Immediately received an automated response. So I sent another follow-up.
• This time, presumably a human responded, since it had a name (but who knows…!?)
• Sadly, there was no compensation ????. However, there was an acknowledgement of the issue, which I found positive and encouraging

<From the OpenAI Team> While our models are designed to handle a wide range of tasks, they are not infallible and can sometimes make mistakes, especially with tasks that may seem straightforward to humans. Your feedback is crucial in helping us identify and address these issues. We have taken note of your comments and will review them internally.

Key Highlights

• ChatGPT spells correctly, but counts incorrectly
• Forcing the model to re-think through challenge questions and examples is helpful
• Can be stubborn while clearly hallucinating. Took several iterations before it relented!
• However once it realized the error, it was surprisingly helpful in creating an argument for compensation with email draft to send to OpenAI
• It was also reasonably creative and admirably self-deprecating in creating a dummy blog article (see the below snapshot for an extract). I loved some of the flourish it included…notice the awesome eye-roll!