OpenAI Doesn't Know Its AI

Since ChatGPT blew up last November, it has caused a state of panic for every high school English teacher who assumes that students are now going to have an AI write all their papers. NY Public schools have banned ChatGPT outright, and others will surely follow.

In response to this (and other use cases where it’s important to know if it is AI generated), OpenAI announced a new AI Text Classifier which can detect if something was written with AI.

This is a difficult problem to solve. It is not like a basic plagiarism checker where you can look at a series of words and see what percentage of them are a close match. Because with AI, it is meant to mimic regular writing. This isn’t like in chess, where it can be clear if an AI is dictating the moves. AI Chess programs aren’t trying to mimic humans, they’re trying to beat them.

OpenAI openly admits that “Our classifier is not fully reliable.” In their testing, they say, “our classifier correctly identifies 26% of AI-written text as ‘likely AI-written,’ while incorrectly labeling human-written text as AI-written 9% of the time.” All in all, they have developed a five-point scale for any English-language text you drop in:

1. Very Unlikely (meaning least likely to be AI-written)
2. Unlikely
3. Unclear
4. Possibly
5. Likely (meaning it is likely to be AI-written)

I wanted to put this to the test. I write a lot, both with and without AI, so I ran some of my work over the past six months through the checker. Here were the buckets.

1)????Human Written works. For this, I used the last five articles I wrote on LinkedIn. Outside of Grammarly for spell/grammar checking, I’m not touching AI when I write these.

Hypothesis: I’d assume these would all come back as very unlikely being AI written.

2)????Hybrid AI/Human writings. In the book I released last week, The Autobiography of George Santos, I used AI to start every one of the fifteen chapters. But I edited those, added sections, etc., and produced the published work.

Hypothesis: My guess is that each of the chapters would fall under unlikely or unclear. Maybe a few where I minimally edited will be presented as possibly.

3)????AI Written works. The book Artificial America was 57 short stories, all 100% written by AI. I didn’t change a word, save for truncating a few for length.

Hypothesis: OpenAI says the tool correctly identifies AI-Written works 26% of the time. I would assume we would have a similar percentage here. These stories are all pure-AI written (and in fact I used OpenAI’s own ChatGPT and GPT-3 to create them.)

Here's the result:

1)?????Human Written works. I ran the AI Text Classifier against these five articles:

The Battle of the AIs: Artificial Intelligence vs. Academic Integrity

What AI Likes About North Korea, and Dislikes About Where You’re From

Weird AI

The Perfect Holiday Gift

What AI Thinks CEOs Look Like

… and it was 5/5. It said each was very unlikely written by AI. That's good; at least my brain hasn't been taken over by our soon-to-be AI overlords.

2)????Hybrid Writings. For the thirteen George Santos chapters I could run (some were shorter chapters, and the model didn’t have enough data), almost all of them were very unlikely. In total, it was 10/13 Very Unlikely, 2/13 Unlikely, and 1/13 Unclear. So by taking an AI writing and heavily editing it, in this instance, the tool will not likely detect the AI.

3)????AI Writings. For Artificial America, I was able to run the model on 56 chapters. Again, these were 100% written by AI. Here are the results with percentages.

• Very Unlikely (12/56) 21%
• Unlikely (26/56) 46%
• Unclear (13/56) 23%
• Possibly (4/56) 7%
• Likely (1/56) 2%

OpenAI claims that this tool caught 26% of AI Written works as likely. But using my AI-Written book, it only caught 2%. That’s a huge gap for what is a decent sample size. Maybe there is something with how I write prompts and the style it is returning that is “less-AI-like.” Maybe OpenAI is just being conservative with the labeling and doesn’t want to have too many false positives. Either way, at least with the texts I put in, the tool seems to be extremely inaccurate.

I wanted to try one last unscientific test. I gave the following prompt to ChatGPT: “write a short article but in the article make it abundantly clear that the article is written by ChatGPT. Don't hide it at all!” And it returned this:

What did OpenAI say about the likelihood that this was written by AI? Possibly.

A reliable AI detection tool still has a long way to go. Have you had similar experiences with the tool? If so, why do you think OpenAI released it when it is clearly not ready? Share your thoughts in the comments!