Are AI Content Detectors Reliable In 2023?

Artificial Intelligence is the new player in the market, in a short span of time AI technology is spreading like crazy. Some recently launched AI tools for many different purposes are super scary!

You won't believe me if I say there's an AI out there that can literally interpret your dreams. You just have to explain your dream in text and the AI will tell you the meaning of it. How cool is that?

This rapid growth of AI has several potential backlashes too, such as misuse of AI content, poor and unhelpful content, copyright issues, plagiarism, etc.

Furthermore, AI content can also be subject to copyright and plagiarism issues, as it can generate text, images, and videos that may infringe on someone else's intellectual property.

To prevent that, many open-source and enterprise-level AI content detectors are designed to detect AI-generated content. However, the question is do they really work? Can you trust them? So, let's find the answer to whether AI content detectors are reliable in 2023.

Well, some of the AI detectors can easily detect the content produced by AI such as chatGPT-3, other AI platforms and even the smart algorithm of chatGPT-4. On the other hand, some of the AI content detectors show mixed and often totally misleading results.

I will experiment with some of the popular AI content detectors available out there and show you the results. Let's start with the most popular AI content detector available out there, the Writer.

Writer

Writer claims that their free detector can check up to 1,500 characters to check whether it's generated using AI or not. Their algorithm seemed okay to me when I first started using it, but later I discovered that their detection algorithm is really messed up. To prove that, I'll be using this whole paragraph section for the test.

As you can see in the image above, I've put the entire paragraph that I wrote to the detector, and it claims it's only 52% human generated. Well, our buddy Writer, that was pretty offensive. I've tasted it from many angles, and unfortunately, 8 out of 10 times it provided me with false detections.

Now, let's test this out using pure chatGPT content and see how our friend performs. For this test, I gave the prompt "Tell me about linked in marketing in short and simple tone" on chatGPT.

I directly pasted the result without any change to the Writer content detector and got the result below.

The result shows it's 25% generated, our buddy has done a pretty good job, but there's a problem. When I slightly changed the sentence, the result drastically changed as follows.

I changed "business-to-business" to "b2b", and the result went straight to 84% human-generated content. That's concerning, right? But that's not the full picture.

It went to straight 91% when I added "of same interest" at the end of the first paragraph. My analysis says it happened because of the grammatical mistake I made in the sentence. It was supposed to be "of the same interest", so "the" was missing. From this, it's evident that Writer's detection algorithm needs polishing.

I'm not yet finished surprising and disappointing you with harsh facts. Did you know that a classifier tool was launched by OpenAI themselves to detect AI generated content? Sounds promising, right? All right, let's rig it.

ChatGPT Classifier

Although chatGPT is the so-called king of the language model-based AI for now, you'll be disappointed to know that their classifier is a botched and faulty tool. To test the classifier, I've used a paraphrased version of this write-up as follows.

I copied and pasted the whole thing unchanged on the chatGPT classifier tool. Guess what? The result is ridiculous. The classifier couldn't detect it as AI-generated content and claimed it was unlikely AI-generated.

Hence, it's evident that the chatGPT tool itself cannot detect its own content. I've conducted this text multiple times with both human and content generated using chatGPT. The results were not satisfying 7 out of 10 times.

Another major drawback of this classifier tool is that sometimes it can't provide any valid result at all. For example, I used the introduction part of this write-up to test the results.

As you can see, the result shows that the tool is "unclear" whether it's human-generated or generated. Many other tools, such as Content Scale, GPT Zero, Crossplag, etc., are available online with faulty detection algorithms and misleading results.

So, which AI content detection tool should you use? I will share some legit AI content detection tools and AI tools to interpret your dreams in my next article (for real). Till then, stay tuned! Goodbye.