AI detectors simply don’t work

Even though they’re widely known not to work, AI detecting software is still on the rise. Even those that work a little bit are getting worse every day as AI continues to improve. I saw a quote on Reddit that summed it up nicely, where a user said:

If AI could ever be used to distinguish between AI text and human text, that same AI would be able to generate human text. It’s a paradox.

Related is this podcast episode from Christopher Penn, appropriately titled “AI Detectors are a Joke”. I’ve done similar tests, running historical documents through some of them, and the results are about the same — most historical documents, like the US Constitution, are shown by these systems as being largely AI-written.

There are still huge challenges for how to handle the rise of AI, particularly when it comes to education, but using any kind of AI detection tool is a fool’s errand, and as Penn says, could lead to “potentially damaging results”.

I don’t have the answer, but these tools certainly aren’t are it.