Popular AI text detection programs frequently make mistakes, even misclassifying historical documents as machine-generated. In experiments, the ZeroGPT service rated the U.S. Declaration of Independence of 1776 as being produced by a neural network with a probability ranging from 95% to 100%, according to Nature magazine.
The issue stems from how these detectors work: they rely on a metric called “perplexity,” which evaluates the predictability of word choices. Texts from neural networks are statistically more predictable, so formal and grammatically correct human speech is often mistakenly labeled as machine-made.
Research shows that for English-language essays, the rate of false positives can reach as high as 16%, and for non-native speakers, this figure climbs to 61%. Moreover, advances in neural networks and the use of “humanizer” programs further confuse detection algorithms.
Experts note that AI detectors differ fundamentally from plagiarism checkers: the latter identify specific sources of copied content, while generative text detectors provide no clear evidence. Human and machine-written text are not mutually exclusive, since AI is trained on billions of human words.
Specialists urge universities not to blindly trust detection technology and to revamp assessment systems, focusing on the transparency of the writing process rather than just the final output. Marzena Karpinska, a linguist and computer scientist at Simon Fraser University, warns: “We absolutely cannot broadly reject people just because an AI detector has labeled their text as generated.”
About the Author
