Rethinking AI-Generated Text Detection: A Strong Baseline and the Distribution-Shift Problem That Remains

• URL: http://arxiv.org/abs/2607.03680v1
• Date: Sat, 04 Jul 2026 03:20:23 GMT
• Title: Rethinking AI-Generated Text Detection: A Strong Baseline and the Distribution-Shift Problem That Remains
• Abstract summary: We show that a plain, fully fine-tuned RoBERTa matches or exceeds the specialized detectors those benchmarks are built around.<n>The remaining challenge is the distribution shift.<n>We identify a key failure mode: under distribution shift, the detector can assign high-confidence machine labels to human-written text.
• Score: 27.078441680572254
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent AI-generated text detection work often introduces a new benchmark together with a specialized detector tailored to it. We revisit this practice from a baseline-first perspective. Across several benchmarks, we show that a plain, fully fine-tuned RoBERTa matches or exceeds the specialized detectors those benchmarks are built around. This suggests that much of the recent architectural complexity is not what drives strong in-distribution detection. The remaining challenge is the distribution shift. The same strong baseline degrades sharply when the topic domain or generating model changes at test time, and simply adding more source data does not close the gap. We identify a key failure mode: under distribution shift, the detector can assign high-confidence machine labels to human-written text from unseen domains. We then study two lightweight domain adaptation methods to address this problem: $K$-shot adaptation with first-order MAML over LoRA adapters, and a per-sample confidence-weighted ensemble built on top of the adapted detector. Overall, our results suggest that progress in AI-generated text detection should be measured not only by in-distribution performance, but also by robustness under distribution shift.

Related papers

• Hitting a Moving Target: Test-Time Adaptation for AI Text Detection under Continual Distribution Shift [7.332470421231734]
  deployed approaches for AI text detection often rely on training-time access to labeled datasets of both human-written and AI-generated text.<n>We propose a test-time adaptation approach, using semi-supervised learning, that adapts to distribution shifts by leveraging homogeneity among unlabeled samples observed at inference time.<n>We establish that test-time adaptation is a promising framework for AI text detection in the wild.
  arXiv  Detail & Related papers  (2026-06-23T20:37:18Z)
• Base Models Look Human To AI Detectors [68.7723287560362]
  We show that generated text from base models is often judged overwhelmingly human, whereas text generated by their instruction-tuned counterparts is not.<n>We propose Humanization by Iterative Paraphrasing (HIP), a detector-agnostic pipeline that minimally fine-tunes a base model into a paraphraser and applies it iteratively.<n>Our findings suggest that current detectors are tracking artifacts of instruction tuning and local context more than any in notion of machine-generated text.
  arXiv  Detail & Related papers  (2026-05-19T08:13:12Z)
• Log-Likelihood, Simpson's Paradox, and the Detection of Machine-Generated Text [0.039998518782208774]
  We show that the token-level signal distinguishing human and machine text is non-uniform across the hidden space of the detector model.<n>We introduce a learned local calibration step grounded in Bayesian decision theory.<n>This single intervention dramatically and consistently improves detection performance across all baseline detectors and all datasets we consider.
  arXiv  Detail & Related papers  (2026-05-07T13:59:07Z)
• Reward-Guided Semantic Evolution for Test-time Adaptive Object Detection [82.2968697030677]
  Open-vocabulary object detection with vision-language models (VLMs) such as Grounding DINO suffers from performance degradation under test-time distribution shifts.<n>We propose Reward-Guided Semantic Evolution (RGSE), a training-free framework that directly refines the text embeddings at test time.
  arXiv  Detail & Related papers  (2026-05-06T06:17:41Z)
• Why AI-Generated Text Detection Fails: Evidence from Explainable AI Beyond Benchmark Accuracy [0.9169660430821364]
  We propose a framework that integrates linguistic feature engineering, machine learning, and explainable AI techniques.<n>Using SHAP-based explanations, we show that the most influential features differ markedly between datasets.<n>We believe that this knowledge helps build AI detectors that are robust across different settings.
  arXiv  Detail & Related papers  (2026-03-24T12:46:15Z)
• Detecting the Machine: A Comprehensive Benchmark of AI-Generated Text Detectors Across Architectures, Domains, and Adversarial Conditions [0.0]
  Existing benchmarks evaluate a single detector on a single dataset under ideal conditions.<n>We present a benchmark evaluating diverse detection approaches across two corpora.<n>Results show that transformer models achieve near-perfect in-distribution performance but degrade under domain shift.
  arXiv  Detail & Related papers  (2026-03-18T09:27:27Z)
• Beyond Raw Detection Scores: Markov-Informed Calibration for Boosting Machine-Generated Text Detection [105.14032334647932]
  Machine-generated texts (MGTs) pose risks such as disinformation and phishing, highlighting the need for reliable detection.<n> Metric-based methods, which extract statistically distinguishable features of MGTs, are often more practical than complex model-based methods that are prone to overfitting.<n>We propose a Markov-informed score calibration strategy that models two relationships of context detection scores that may aid calibration.
  arXiv  Detail & Related papers  (2026-02-08T16:06:12Z)
• DetectAnyLLM: Towards Generalizable and Robust Detection of Machine-Generated Text Across Domains and Models [60.713908578319256]
  We propose Direct Discrepancy Learning (DDL) to optimize the detector with task-oriented knowledge.<n>Built upon this, we introduce DetectAnyLLM, a unified detection framework that achieves state-of-the-art MGTD performance.<n>MIRAGE samples human-written texts from 10 corpora across 5 text-domains, which are then re-generated or revised using 17 cutting-edge LLMs.
  arXiv  Detail & Related papers  (2025-09-15T10:59:57Z)
• Group-Adaptive Threshold Optimization for Robust AI-Generated Text Detection [58.419940585826744]
  We introduce FairOPT, an algorithm for group-specific threshold optimization for probabilistic AI-text detectors.<n>We partitioned data into subgroups based on attributes (e.g., text length and writing style) and implemented FairOPT to learn decision thresholds for each group to reduce discrepancy.<n>Our framework paves the way for more robust classification in AI-generated content detection via post-processing.
  arXiv  Detail & Related papers  (2025-02-06T21:58:48Z)
• Robust AI-Generated Text Detection by Restricted Embeddings [6.745955674138081]
  We focus on robustness of detectors of AI-generated text, namely their ability to transfer to unseen generators or semantic domains. We show that clearing out harmful linear subspaces helps to train a robust classifier, ignoring domain-specific spurious features. Our best approaches for head-wise and coordinate-based subspace removal increase the mean out-of-distribution (OOD) classification score by up to 9% and 14% in particular.
  arXiv  Detail & Related papers  (2024-10-10T16:58:42Z)
• Hard-normal Example-aware Template Mutual Matching for Industrial Anomaly Detection [78.734927709231]
  Anomaly detectors are widely used in industrial manufacturing to detect and localize unknown defects in query images.<n>These detectors are trained on anomaly-free samples and have successfully distinguished anomalies from most normal samples.<n>However, hard-normal examples are scattered and far apart from most normal samples, and thus they are often mistaken for anomalies by existing methods.
  arXiv  Detail & Related papers  (2023-03-28T17:54:56Z)
• Unsupervised and self-adaptative techniques for cross-domain person re-identification [82.54691433502335]
  Person Re-Identification (ReID) across non-overlapping cameras is a challenging task. Unsupervised Domain Adaptation (UDA) is a promising alternative, as it performs feature-learning adaptation from a model trained on a source to a target domain without identity-label annotation. In this paper, we propose a novel UDA-based ReID method that takes advantage of triplets of samples created by a new offline strategy.
  arXiv  Detail & Related papers  (2021-03-21T23:58:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.