Do AI Detectors Work? What to Know

As more people use AI to write, a second industry has popped up promising to catch them: AI detectors. Teachers, editors, and managers are leaning on these tools to decide whether text was written by a human. The problem is that the tools do not work as well as they claim, and treating them as proof can hurt innocent people.

What is an AI detector?

An AI detector is a tool that reads a piece of text and guesses how likely it was written by an AI like ChatGPT. It looks for patterns that AI writing tends to have, such as very even, predictable phrasing, and returns a score or a verdict like “85 percent AI.” The key word is guesses.

Do AI detectors work?

Not reliably. Vendors advertise accuracy in the high nineties, but independent tests keep landing much lower. Here is the gap.

What is claimed vs what tests show	Result
Vendor accuracy claims	95 to 99 percent
Independent test results	roughly 65 to 88 percent
OpenAI’s own detector (2023)	caught about 26 percent; shut down
Short text under 50 words	accuracy collapses

Why do AI detectors flag human writing?

Because clear, simple, well-structured writing looks “predictable” to a detector, and that is exactly what good human writing often is. The result is false positives, where real human work gets labeled as AI. A Stanford study found detectors flagged about 61 percent of essays by non-native English speakers as AI-generated, even though people wrote every word. That bias alone should make anyone cautious about using these tools to judge students or job applicants.

Did OpenAI make an AI detector?

Yes, and the story is telling. In 2023 OpenAI, the maker of ChatGPT, released a tool to detect AI-written text, then retired it months later because it was not accurate enough. When the company that builds the AI cannot reliably detect its own output, that is a strong hint about the whole category.

Can you trick an AI detector?

Easily, which is the other half of the problem. Light editing, rephrasing, or running text through another tool can flip a detector’s verdict. So detectors punish careful writers who happen to write cleanly, while anyone trying to hide AI use can often slip past. That is the worst of both outcomes.

What should teachers and students do instead?

• Treat a detector score as a weak signal at most, never as proof.
• Look at the writing process, drafts, version history, and the ability to explain the work in person.
• Design assignments that ask for personal experience, in-class steps, or reflection that AI cannot fake well.
• Have a calm conversation before any accusation; a single score cannot support one.

What does this mean for using AI well?

If that idea resonates, read how to use AI without losing your edge and the rest of our perspectives on living and working with AI. For the bigger picture on building real skill, see our take on AI literacy.

Common questions about AI detectors

Are AI detectors accurate?

Not accurate enough to trust on their own. Independent tests put them well below vendor claims, and they produce real false positives. Use them as a hint at most.

Can a teacher prove I used AI with a detector?

No. A detector score is not proof. Given the documented false-positive rates, a fair process looks at drafts, history, and a conversation, not a single number.

Why was a human essay flagged as AI?

Because clean, simple writing can look “predictable” to a detector. This hits non-native English speakers especially hard, as a Stanford study showed.

Is there a detector that always works?

No. As AI writing improves and text can be lightly edited, no detector can promise reliable results. Be wary of any tool claiming it can.

Sources

• OpenAI, on its AI text classifier (later discontinued)
• Stanford HAI, AI detectors and bias against non-native English writers

You may also like

• How to use AI without losing your edge
• What AI literacy really means
• Perspectives on living with AI
• What is ChatGPT? A simple guide
• AI and your kid’s reading
• How to use Claude AI
• ChatGPT pricing explained