ChatGPT's Veracity Problem

ChatGPT is getting all the headlines these days, and as a system that is built on GPT-3, it’s got a problem with the truth. It’s not that it lies, but it’s composing based on what it has ingested, which may or may not be true. What it seems to be very good at is generating compositions that sound reasonable, what journalists might call, “truthiness.” Its output usually sounds truthy, and sometimes that is good enough.

But when facts are required, the current publicly available Natural Language Generation (NLG) engines often falter. Take for example the CNET fiasco. The technology blog has been piloting a generative AI tool built internally by publishing articles. Some articles had huge errors on the very subjects they were about. But they sounded great.

There were simple but profound mistakes that anyone with a passing familiarity with the topic would catch, but somehow they made it to publication. Running this article through the standard newsroom process of editing and fact-checking — by human subject-matter experts — would have caught the errors right away, so this was a miss on the machine and human sides of the process. Anyone creating content with or without AI should be fact-checking their content anyway.

ChatGPT and other popular NLG engines like Jasper, and Anyword are all built on GPT-3. As such, they share what I call the veracity problem. And since GPT-3 is a neural network, it’s not easy (maybe even impossible), to determine where the fact or figure came from. Which makes it very hard for everyday users to teach the systems to stop making these kinds of errors.?

I think these systems have other shared problems such as sometimes misunderstanding concepts to do with numbers, and occasionally outputting plagiarized results. Which requires other tools like plagiarism checkers — a different AI to check the AI, oh boy.

Human in the loop! With AI system this is standard best practice, and since we’re using NLG to speak to humans, let’s please keep humans involved in reviewing and approving what is created before it’s shared and acted upon.

This technology is in its infancy and there is still a ways to go before it’s ready for prime time.?

With that in mind, we should all have fun with it and figure out where it can help us the most.

* While I do use ChatGPT among other AI tools, this post was written the old-fashioned way.