ChatGPT does not reverse text or count words (with examples!)
Daniel Ostromich
Experienced Data Leader with over 8 years in health information systems and clinical operations.
I was working on a project with ChatGPT open in one tab and stopped to take a break. At the start of the break I remembered how I tested ChatGPT on reversing nonsense text and it couldn't quite get it right through multiple iterations. Please note that the below image is from July 17, 2023 using GPT-4. The cover image is the result from today (Aug 03, 2023). For whatever reason the answer is delivered as "code" and is still incorrect.
Another "feature" of ChatGPT is that it cannot count words very well. In this case I pasted a corpus of text with 1509 words. below you can see some of the different counts given. As always, the model is GPT-4.
I did some testing with HuggingChat, but did not include examples as it is was much less sophisticated than ChatGPT. Unfortunately I did not have access to Claude 2 and Bard as I am in Canada.
I'd love to see in the comments what other failings you may have personally found with LLMs.