ChatGPT Isn't Creepy, It's Just Dumb

(Bonus points for commenting on why I chose this image.)

There's been an enormous amount of buzz these last few months about ChatGPT, an AI model trained to respond to user queries in a conversational fashion. Many stories have been written more recently about how it occasionally goes off the rails in "disturbing" ways.

I understand why people get weirded out by some of those examples, but it's important to understand what these chatbots are actually doing. That will help us put it in perspective, while also informing our choices as appropriate (and reckless) uses of this emerging technology.

Chatbots like ChatGPT and Bard are not sentient.

They're not thinking the way you or I think. In fact, you could argue that they have no idea what they're saying. Do your eyes deceive you? How can that be when you asked it a question and it responded with a unique, cogent reply?

Let's look behind the curtain of this technological illusion. (Magic & mentalism is a lifelong passion of mine so I'm tickled by this analogy.)

AI models have been referred to as "prediction engines". You might call them "guessing machines". In the case of an AI chatbot, it's been trained on hundreds of billions of words from around the internet. Trained to do what exactly? Natural language processing and pattern recognition.

ChatGPT analyzes the user's question or statement for content and intent. It then looks for patterns and associations in its database that relate to the user's question or statement, and uses those patterns to generate a response. It's very good at mimicking the human language, which is why the output appears to us like a thoughtful response. The overall experience can feel like a human conversation because preceding context is retained and utilized.

So, while it may "feel" like a human conversation, ChatGPT is not actually experiencing or understanding the concepts it's discussing the way a human would. That feeling is an illusion.

As with any series of estimations, you can expect that it might deviate over time. Kind of like trying to walk straight through a large forest without a compass. The longer your "conversation" with the chatbot is (if not steered skillfully), the more likely it is to go off course and end up way in left field.

If the user's choice of topic is something personal, then the output may even end up somewhere "disturbing". This isn't reason to panic about a "creepy machine" with ill intent. It's simply proof that AI models have appropriate as well as inappropriate use cases. A carpenter doesn't use a level to hammer in a nail. Likewise, we shouldn't use ChatGPT as some sort of romantic companion.

We're not chatting with a sentient, intelligent organism and so we shouldn't treat it as one. Nor should we trust blindly in the answers it generates, as other stories (and embarrassing public demos) have demonstrated. As incredibly capable and undeniably impressive as the technology is, there's danger in giving it more credit than it's due. It's here now. Let's use it wisely.

Let me know in the comments if you found this informative. Does this change the way you will use chatbots like ChatGPT? (Bonus points for correctly guessing why I chose this particular lightbulb image.)