I Love a Good Riddle

Putting AI Models to the Test: Asking ChatGPT Common Sense Questions?

I love a good riddle.

My favorite is one that I heard years ago.

What can go up a chimney down but cannot go down a chimney up?

Early chatbot based on Large Language Models (LLMs) struggled to solve simple riddles like this.

AI systems often lacks robust models of the real world.

These systems often don’t understand how mechanical, economic, and social systems work.? They are unsure how people relate to each other.? And are missing basic expectations of what is reasonable and what is not.

AI cannot understand context and personal relationships.

Human relationships are complex and nuanced, layered upon previous experiences.? AIs may struggle to understand context and how it impacts an appropriate response.

AI can misunderstand cause and effect.

AI is really good at identifying patterns and making predictions based on data, it seems to struggle at times to understand the reason behind human behavior.?

AI struggles with abstract concepts and symbolic reasoning.

Primarily focused on pattern matching and data analysis, AI may be limited in its ability to engage with abstract concepts like creativity.? It can also struggle with symbolic reasoning, often demonstrated by its inability to answer simple riddles.

AI lacks common sense and general knowledge.

As my dad used to say “Common sense, not to common.”

AI now has access to and can process lots of data.? But data alone doesn’t make up for common sense and the general knowledge that humans use to navigate the world and understand complex questions.

When it comes to AI capabilities versus true intelligence, LLM vendors are making great strides to solving this problem. ?I recently asked two different LLMs the following questions:

A banana flies like fruit, but fruit flies like apples.

OpenAI’s ChatGPT 4o responded:

That's a fun play on words! It flips the meaning of "flies" between a verb (to fly) and a noun (fruit flies). The sentence plays with both meanings to create a quirky, clever twist. If you ever need more wordplay or want to explore puns further, just let me know!

Nvidia’s llama-3.1-nemotron-70b-instruct responded:

A classic example of a linguistic trick known as a "semantic ambiguity" or "wordplay"!

Let's break down the sentence to appreciate its cleverness:

"A banana flies like fruit"

• Here, "flies" is a verb, meaning to move through the air. The sentence is saying that a banana (a type of fruit) moves through the air in a manner similar to how fruit (in general) might move through the air (e.g., if it were thrown or blown by the wind).

"but fruit flies like apples"

• Now, "fruit flies" is a noun phrase, referring to a type of small insect (Drosophila) that is attracted to fruit.
• "like" is a preposition, indicating a preference or attraction.
• So, this part of the sentence is saying that fruit flies (the insects) are attracted to apples.

The wordplay relies on the multiple meanings of:

1. "flies" (verb vs. noun)
2. "like" (manner of movement vs. preference)
3. "fruit flies" (a type of fruit's movement vs. a type of insect)

This clever sentence is often used to illustrate the complexities and nuances of the English language, and how a simple change in context or word meaning can completely flip the interpretation of a sentence!

Exposing the limitations of AI is more difficult today than it has ever been.

Carefully designed questions will help us better understand the current state of AI.? It could even help us determine if and when an AI becomes sentient.? And, it might help in preventing misuse.

An AI Turing Test is more relevant today than ever.

The Turing Test, created by Alan Turing, measured a machine's ability to exhibit intelligent behavior indistinguishable from a human. ?To update it for modern AIs, we will need to include tasks beyond conversation, like problem-solving, creativity, and emotional understanding.? This will test whether AI can perform similarly to humans in real-world situations.

It’s already increasing hard to tell when we’re talking to a bot.? Even if you ask “Are you human?”, we might be lied to.

How will we tell?

Will it matter?

Next time you’re interacting with an AI, consider asking “Is there anything else I should know?”.

If you’re read this far and still don’t know the answer to the riddle, it’s an umbrella.

#ArtificialIntelligence #NvidiaAI #ChatGPT #Kognetiks #Chatbot

Was this helpful?? ???Repost if it resonates.? Follow Stephen Howell for more.