Hitting a Moving Target: Test-Time Adaptation for AI Text Detection under Continual Distribution Shift

• URL: http://arxiv.org/abs/2606.25152v1
• Date: Tue, 23 Jun 2026 20:37:18 GMT
• Title: Hitting a Moving Target: Test-Time Adaptation for AI Text Detection under Continual Distribution Shift
• Abstract summary: deployed approaches for AI text detection often rely on training-time access to labeled datasets of both human-written and AI-generated text.<n>We propose a test-time adaptation approach, using semi-supervised learning, that adapts to distribution shifts by leveraging homogeneity among unlabeled samples observed at inference time.<n>We establish that test-time adaptation is a promising framework for AI text detection in the wild.
• Score: 7.332470421231734
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deployed approaches for AI text detection often rely on training-time access to labeled datasets of both human-written and AI-generated text. This approach is vulnerable to three types of distribution shifts that occur continually post-deployment, and for which labeled data is often unavailable: adversarial humanization, new LLMs being released, and temporal drift in human writing. Simultaneously, existing approaches do not leverage a key signal of LLM usage: inference-time homogeneity. We propose a test-time adaptation (TTA) approach, using semi-supervised learning, that adapts to distribution shifts by leveraging homogeneity among unlabeled samples observed at inference time. Empirically, we find that state-of-the-art supervised detectors systematically fail when they encounter distribution shifts in AI-generated and human writing, both adversarial and natural, while test-time adaptation with semi-supervised learning is largely robust; e.g., the commercial model Pangram detects just 24.1% of our adversarial AI-generated text, compared to 90.5% for our test-time approach. We establish that test-time adaptation is a promising framework for AI text detection in the wild. We publicly release our code (which includes code for model training, evaluation, and plots) at https://github.com/kkr36/llm_detection.

Related papers

• Rethinking AI-Generated Text Detection: A Strong Baseline and the Distribution-Shift Problem That Remains [27.078441680572254]
  We show that a plain, fully fine-tuned RoBERTa matches or exceeds the specialized detectors those benchmarks are built around.<n>The remaining challenge is the distribution shift.<n>We identify a key failure mode: under distribution shift, the detector can assign high-confidence machine labels to human-written text.
  arXiv  Detail & Related papers  (2026-07-04T03:20:23Z)
• AEyeDE: An Attention-Based Attribution Framework for AI-Generated Text Detection [1.373282478189168]
  We propose textscAEyeDE, an attribution-driven approach to human-AI authorship detection.<n>We show that attention-based attribution maps provide a complementary and interpretable signal for AI-generated text detection.
  arXiv  Detail & Related papers  (2026-04-13T19:30:40Z)
• Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection [71.59834293521074]
  We develop a framework to distinguish between human-authored and machine-generated text.<n>Our method achieves 98.3% AUROC and AUPR with only 8.9% FPR95 on DeepFake dataset.<n>Code, pretrained weights, and demo will be released.
  arXiv  Detail & Related papers  (2025-10-07T08:14:45Z)
• Fine-Grained Detection of AI-Generated Text Using Sentence-Level Segmentation [3.088244520495001]
  A sentence-level sequence labeling model proposed to detect transitions between human- and AI-generated text.<n>Our model combines the state-of-the-art pre-trained Transformer models, incorporating Neural Networks (NN) and Conditional Random Fields (CRFs)<n>The evaluation is performed on two publicly available benchmark datasets containing collaborative human and AI-generated texts.
  arXiv  Detail & Related papers  (2025-09-22T14:22:55Z)
• Group-Adaptive Threshold Optimization for Robust AI-Generated Text Detection [58.419940585826744]
  We introduce FairOPT, an algorithm for group-specific threshold optimization for probabilistic AI-text detectors.<n>We partitioned data into subgroups based on attributes (e.g., text length and writing style) and implemented FairOPT to learn decision thresholds for each group to reduce discrepancy.<n>Our framework paves the way for more robust classification in AI-generated content detection via post-processing.
  arXiv  Detail & Related papers  (2025-02-06T21:58:48Z)
• Time-series Generation by Contrastive Imitation [87.51882102248395]
  We study a generative framework that seeks to combine the strengths of both: Motivated by a moment-matching objective to mitigate compounding error, we optimize a local (but forward-looking) transition policy. At inference, the learned policy serves as the generator for iterative sampling, and the learned energy serves as a trajectory-level measure for evaluating sample quality.
  arXiv  Detail & Related papers  (2023-11-02T16:45:25Z)
• Preserving Fairness in AI under Domain Shift [15.820660013260584]
  Existing algorithms for ensuring fairness in AI use a single-shot training strategy. We develop an algorithm to adapt a fair model to remain fair under domain shift.
  arXiv  Detail & Related papers  (2023-01-29T06:13:40Z)
• TeST: Test-time Self-Training under Distribution Shift [99.68465267994783]
  Test-Time Self-Training (TeST) is a technique that takes as input a model trained on some source data and a novel data distribution at test time. We find that models adapted using TeST significantly improve over baseline test-time adaptation algorithms.
  arXiv  Detail & Related papers  (2022-09-23T07:47:33Z)
• DAAIN: Detection of Anomalous and Adversarial Input using Normalizing Flows [52.31831255787147]
  We introduce a novel technique, DAAIN, to detect out-of-distribution (OOD) inputs and adversarial attacks (AA) Our approach monitors the inner workings of a neural network and learns a density estimator of the activation distribution. Our model can be trained on a single GPU making it compute efficient and deployable without requiring specialized accelerators.
  arXiv  Detail & Related papers  (2021-05-30T22:07:13Z)
• A Novel Anomaly Detection Algorithm for Hybrid Production Systems based on Deep Learning and Timed Automata [73.38551379469533]
  DAD:DeepAnomalyDetection is a new approach for automatic model learning and anomaly detection in hybrid production systems. It combines deep learning and timed automata for creating behavioral model from observations. The algorithm has been applied to few data sets including two from real systems and has shown promising results.
  arXiv  Detail & Related papers  (2020-10-29T08:27:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.