AI writing tells persist despite model updates

Overview

As large language models become increasingly sophisticated, distinguishing AI-generated text from human writing remains a persistent challenge for researchers and readers alike. While early indicators such as excessive em-dash usage and overused verbs like "delve" have largely been stamped out by newer model iterations, a new study reveals that these tells are being replaced by different patterns rather than disappearing entirely.

Research conducted by the marketing firm Graphite indicates that frontier AI models still exhibit distinct linguistic habits that set them apart from human authors. These subtle quirks range from specific word choices to complex sentence structures, suggesting that while models are becoming more natural, they have not yet achieved true stylistic neutrality. The findings highlight a cat-and-mouse dynamic between AI developers aiming for human-like fluency and the inherent statistical biases embedded in their architectures.

The Evolution of AI Writing Tells

The study examined writing habits across several major frontier models, identifying thousands of phrases that appeared with significantly higher frequency in AI-generated content than in human samples. Graphite defined a "tell" as any phrase or word combination appearing at least twice as often in AI text compared to human text, ultimately cataloging approximately 13,000 such indicators.

Notably, the research found that some older, widely recognized tells have been successfully eliminated. For instance, OpenAI’s latest model, Astra, now uses em-dashes 88 percent less often than human samples. Similarly, Gemini 3.1 Pro has almost completely removed the punctuation mark from its output. This suggests that when specific tells become common knowledge, developers can effectively train models to suppress them.

However, the elimination of one tell often leads to the emergence of another. The overall volume of detectable patterns has remained surprisingly steady. According to Greg Druck, Graphite’s chief AI officer, the labs are succeeding in removing well-known artifacts, but new ones consistently pop up with each model update. Every version of every model tends to display its own unique set of quirks, making it difficult to establish a single, universal marker for AI generation.

Model-Specific Quirks and Patterns

Different AI models exhibit distinct stylistic fingerprints, even when tasked with the same objective. Graphite’s analysis of Claude Opus 5.5 revealed that its most prominent tell is the word dependable, which appears 23 times more frequently than in human writing. While the model has moved away from the classic "it’s not X, it’s Y" construction, it still favors a variation: stating that something "is more than an X, it’s a Y."

"It turns out that Claude models are actually getting closer to the human word distribution over time. And for the GPT models, it’s getting further away." — Greg Druck, Graphite Chief AI Officer

Opus 5.5 also demonstrates a strong propensity to explain the significance of topics. The phrase "this matters" appears 116 times more often in its output than in human texts, while "why X matters" occurs 92 times more frequently. This meta-commentary on importance appears to be a deeply ingrained habit that persists despite efforts to streamline the prose.

In contrast, OpenAI’s Astra model relies on a different set of tip-offs. It frequently describes a topic as having "another dimension" and tends to hedge claims by suggesting actions "may provide" or "can provide" specific benefits. Its most significant tell is what Graphite terms corrective framing, where a concept is defined as "not simply X" or presented as an alternative to "relying on X." These constructions were found to be more than 100 times more common in Astra-generated prose than in human writing.

Implications for AI Detection and Development

The persistence of these tells raises questions about the efficacy of current efforts to make AI indistinguishable from human writing. Major labs have publicly touted the naturalness of their latest models. Anthropic’s release notes for Opus 5.5 claimed it "communicates more naturally," while OpenAI stated that users of its GPT-6 versions could expect "more clarity" and "fewer odd turns of phrase."

Despite these claims, Druck is skeptical about the extent to which labs can fully eradicate telltale constructions. He hypothesizes that the sheer scale of these models—containing billions of parameters—makes complete control difficult. With a finite number of tests and evaluations, certain statistical tendencies inevitably slip through the cracks.

"These are giant models with billions of parameters. They have some finite number of tests they can run, and things slip through."

The methodology behind the study also warrants attention. To minimize source bias, researchers used a corpus of 10,000 articles published before the release of ChatGPT as a human baseline. They then had various AI models rewrite summaries of these articles. This approach allowed for a direct comparison of word frequency and sentence construction patterns, providing a robust dataset for identifying systematic differences between human and machine-generated text.

While individual tells may shift with each update, the fundamental divergence in style between human and AI writing remains. For now, these persistent linguistic markers serve as a reminder that even the most advanced language models retain statistical echoes of their training data and architectural design.