AI Detectors Flag Human Writer's Work

Ghostwriter Sally McGraw received an email stating that multiple artificial intelligence detection tools identified her writing sample as machine-generated. McGraw, who has spent more than 20 years as a professional nonfiction writer, had not used AI to create the text. She described feeling angry and embarrassed, comparing her reaction to a cartoon character turning red from the chin to the crown of the head. Although she assured the client the work was hers, she did not press the issue because she considered the job minor. The client confirmed screening the work for AI but stated that experience and fit were the primary reasons for hiring another writer. McGraw was paid for the sample but believes the detector results cost her the opportunity to write the full book proposal.

The rise in AI-generated content has pressured editors to manage increasing submission volumes. Neil Clarke, publisher and editor of the science-fiction magazine Clarkesworld, noted that submissions to the publication doubled from roughly 1,100 per month in 2021 to a higher volume by February 2023. This spike followed the widespread availability of AI agents like ChatGPT, Claude, and Gemini. Clarke temporarily closed the submission portal due to the influx and estimates his own workload increased by about 30 percent. He described the situation as introducing an element of mistrust into the editing process.

Researchers have found that different AI detection tools often reach contradictory conclusions about the same text. A study published in June in the International Journal for Educational Integrity tested 160 academic papers using four widely used detectors. The papers included entirely human-written work, entirely AI-written work, combined text, and AI material rewritten by humans. The study found that Pangram was the most consistently accurate, correctly identifying AI use in 37 of 40 papers containing combined text and 37 of 40 papers where humans rewrote AI-generated text. Max Spero, CEO of Pangram, acknowledged in an emailed statement that the detector can make mistakes, particularly when evaluating short pieces of writing.

Chris Callison-Burch, a professor of computer and information science at the University of Pennsylvania, explained that many detectors rely on large language models to assess how predictable a passage is. These models often find text from their training data unusually predictable, which can lead to false positives for previously published work. Science blogger Janelle Shane experienced this in late 2022 when a detector reported a 98.7 percent confidence that a passage from her 2019 book, You Look Like a Thing and I Love You, was machine-generated. Shane called the result a clear false positive. Callison-Burch noted that unpublished drafts might avoid this specific training data issue but could still be flagged based on writing style patterns.

Detectors look for clues in word choice, punctuation, and sentence structure. Callison-Burch said that the word delve, frequent use of em dashes, and ideas grouped in lists of three can trigger an AI-positive result. Opening sentences with adverbs such as notably or crucially can also raise flags. He emphasized that detectors weigh patterns across an entire piece, so a single em dash or specific word does not necessarily indicate bot usage. The Authors Guild stated in May that the use of AI detectors creates an environment reminiscent of witch hunts, where writers worry about false accusations and how to defend themselves. The organization warned that a false positive can cost an author their publishing contract and reputation.

McGraw says the fear of another accusation has changed her writing habits. She now avoids certain words and sentence constructions and occasionally omits commas she previously included. She hopes this inconsistency will help prove a human wrote the text. McGraw expressed frustration at missing her em dash, a punctuation mark she learned in journalism school and used for decades. She sees an irony in large language models learning from human prose while writers feel pressure to make their work look less like AI. She described the situation as trying to fool a machine into thinking she is not a machine.

Clarke does not personally use AI detectors to screen incoming manuscripts but believes certain tools could help overwhelmed editors manage volume if they verify the findings. He cautioned against turning suspicions into accusations that could damage a writer's reputation. He stated that submissions should end in rejection, not an accusation. McGraw remains braced for a future assignment where the outcome makes or breaks her career, worrying about facing another accusation when a bigger opportunity is on the line.