AI News Research

New training method helps AI text detectors handle changing writing

MIT and Cornell researchers report a test-time adaptation method that helps AI text detectors resist disguised text and shifting writing styles.

Automated summary. Source linked.

Researchers from MIT and Cornell University report that AI text detectors, which are AI models themselves, can be fooled by machine-written text disguised to look human. Detectors can also miss AI text when writing habits shift over time or when new LLMs appear. The team's new paper describes a training technique meant to keep detectors accurate and resilient as writing changes.

The method is called test-time adaptation. It lets a deployed detector learn from new, unlabeled text by adjusting itself, without retraining from scratch. In experiments, a commercial detector identified about 27 percent of disguised AI text, while the new approach reached 93 percent accuracy. In tests mimicking shifts in human writing over time, the adaptive detector reached 85 percent accuracy, versus 73 percent for older methods.

Kevin Ren, a graduate student at Cornell Tech, is the lead author, with Manish Raghavan of MIT and senior author Nikhil Garg of Cornell Tech. The team's work is slated for the Conference on Neural Information Processing Systems. The researchers also caution that the approach may not always work as future LLMs produce more natural-sounding language, making verifiable human text harder to find for training.

Key facts

  • The authors say AI detectors can be tricked by AI text disguised as human writing.
  • Test-time adaptation lets a deployed detector learn from unlabeled new text without full retraining.
  • A commercial detector caught only about 27 percent of disguised AI text in experiments.
  • On natural shifts over time, the adaptive detector scored 85 percent accuracy against 73 percent for traditional methods.
  • Kevin Ren, a Cornell Tech graduate student, is the study's lead author.

Why it matters

Educators, editors, publishers and hiring managers could use it to flag LLM-generated writing that people present as human-written. Cybersecurity experts could also apply it against email phishing and search engine optimization fraud. Pangram supplied API credits, which the researchers used to run its AI text detectors. The team plans to explore other domains where the method could be useful.

Source: Read it at news.mit.edu (opens in a new tab)

More in this topic