Why AI detectors get it wrong so often

01How a detector actually decides

Most detectors are classifiers trained on examples of human and machine text. They look at statistical properties of the writing: perplexity, roughly how surprising each word is given the previous ones, and burstiness, how much sentence length and complexity vary across a passage.

Model output tends to score low on perplexity and low on burstiness, because sampling favours probable continuations and produces even, regular prose. The detector reports the probability that a passage came from a model, and that number is an estimate about a style, not evidence about an author.

02Why the errors are not random

Any writing that is clear, even and conventional scores as machine-like. Technical documentation, legal drafting, academic abstracts and instructional writing all share those properties because the genre demands them.

The best-documented harm falls on people writing in a second language, who often use simpler constructions and more predictable vocabulary — exactly the profile a detector reads as artificial. Studies have repeatedly found substantially higher false-positive rates for non-native English writers, which makes detector output an unfair basis for an accusation.

The reverse error is just as easy. Asking a model to vary its sentence structure lowers a detector's confidence considerably, so the people most motivated to evade are the least likely to be caught.

03How watermarking differs

A watermark is deliberately placed at generation time by the party running the model, and detected with a key. Where present and where the key holder runs the check, it is far more reliable than a classifier, because it is looking for a signal it planted rather than guessing from style.

The limits are different too. A watermark cannot exist in text produced by a model that does not implement one, and detection keys are generally not public, so third parties cannot run the check themselves. A classifier will happily give anyone a verdict on any text, which is precisely why unreliable verdicts are so widespread.

Questions