Why AI-Generated Text Detection Fails: Evidence from Explainable AI Beyond Benchmark Accuracy

• URL: http://arxiv.org/abs/2603.23146v1
• Date: Tue, 24 Mar 2026 12:46:15 GMT
• Title: Why AI-Generated Text Detection Fails: Evidence from Explainable AI Beyond Benchmark Accuracy
• Abstract summary: We propose a framework that integrates linguistic feature engineering, machine learning, and explainable AI techniques.<n>Using SHAP-based explanations, we show that the most influential features differ markedly between datasets.<n>We believe that this knowledge helps build AI detectors that are robust across different settings.
• Score: 0.9169660430821364
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The widespread adoption of Large Language Models (LLMs) has made the detection of AI-Generated text a pressing and complex challenge. Although many detection systems report high benchmark accuracy, their reliability in real-world settings remains uncertain, and their interpretability is often unexplored. In this work, we investigate whether contemporary detectors genuinely identify machine authorship or merely exploit dataset-specific artefacts. We propose an interpretable detection framework that integrates linguistic feature engineering, machine learning, and explainable AI techniques. When evaluated on two prominent benchmark corpora, namely PAN CLEF 2025 and COLING 2025, our model trained on 30 linguistic features achieves leaderboard-competitive performance, attaining an F1 score of 0.9734. However, systematic cross-domain and cross-generator evaluation reveals substantial generalisation failure: classifiers that excel in-domain degrade significantly under distribution shift. Using SHAP- based explanations, we show that the most influential features differ markedly between datasets, indicating that detectors often rely on dataset-specific stylistic cues rather than stable signals of machine authorship. Further investigation with in-depth error analysis exposes a fundamental tension in linguistic-feature-based AI text detection: the features that are most discriminative on in-domain data are also the features most susceptible to domain shift, formatting variation, and text-length effects. We believe that this knowledge helps build AI detectors that are robust across different settings. To support replication and practical use, we release an open-source Python package that returns both predictions and instance-level explanations for individual texts.

Related papers

• Rethinking AI-Generated Text Detection: A Strong Baseline and the Distribution-Shift Problem That Remains [27.078441680572254]
  We show that a plain, fully fine-tuned RoBERTa matches or exceeds the specialized detectors those benchmarks are built around.<n>The remaining challenge is the distribution shift.<n>We identify a key failure mode: under distribution shift, the detector can assign high-confidence machine labels to human-written text.
  arXiv  Detail & Related papers  (2026-07-04T03:20:23Z)
• 'Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions [50.1119556361129]
  We systematically define various notions of AI-generated text and their characteristics.<n>AITDNA is a new benchmark of human-machine co-constructed texts.<n>We benchmark various machine-generated text detectors and find that they often only perform well for specific notions but not as broad detectors.
  arXiv  Detail & Related papers  (2026-06-03T14:05:14Z)
• Show, Don't TELL: Explainable AI-Generated Text Detection [0.42970700836450487]
  We develop a novel architecture, TELL, that bakes explainability from the ground-up.<n>We train TELL on a custom SFT dataset of domain-specific authorship annotations.<n>We report a high (mean 72.3%) win-rate on annotation concreteness, falsifiability, coherence, plausibility and grounding.
  arXiv  Detail & Related papers  (2026-05-27T03:47:25Z)
• Base Models Look Human To AI Detectors [68.7723287560362]
  We show that generated text from base models is often judged overwhelmingly human, whereas text generated by their instruction-tuned counterparts is not.<n>We propose Humanization by Iterative Paraphrasing (HIP), a detector-agnostic pipeline that minimally fine-tunes a base model into a paraphraser and applies it iteratively.<n>Our findings suggest that current detectors are tracking artifacts of instruction tuning and local context more than any in notion of machine-generated text.
  arXiv  Detail & Related papers  (2026-05-19T08:13:12Z)
• DependencyAI: Detecting AI Generated Text through Dependency Parsing [10.075606234222963]
  We introduce DependencyAI, a simple and interpretable approach for detecting AI-generated text.<n>Our method achieves competitive performance across monolingual, multi-generator, and multilingual settings.
  arXiv  Detail & Related papers  (2026-02-17T11:42:28Z)
• Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection [71.59834293521074]
  We develop a framework to distinguish between human-authored and machine-generated text.<n>Our method achieves 98.3% AUROC and AUPR with only 8.9% FPR95 on DeepFake dataset.<n>Code, pretrained weights, and demo will be released.
  arXiv  Detail & Related papers  (2025-10-07T08:14:45Z)
• Text-ADBench: Text Anomaly Detection Benchmark based on LLMs Embedding [27.02879006439693]
  This work performs a comprehensive empirical study and introduces a benchmark for text anomaly detection.<n>Our work systematically evaluates the effectiveness of embedding-based text anomaly detection.<n>By open-sourcing our benchmark toolkit, this work provides a foundation for future research in robust and scalable text anomaly detection systems.
  arXiv  Detail & Related papers  (2025-07-16T14:47:41Z)
• IPAD: Inverse Prompt for AI Detection -- A Robust and Explainable LLM-Generated Text Detector [11.112793289424886]
  Large Language Models (LLMs) have attained human-level fluency in text generation, which complicates the distinction between human-written and LLM-generated texts.<n>Existing detectors exhibit poor robustness on out-of-distribution (OOD) data and attacked data, which is critical for real-world scenarios.<n>We propose IPAD (Inverse Prompt for AI Detection), a novel framework consisting of a Prompt that identifies predicted prompts that could have generated the input text, and two Distinguishers that examine the probability that the input texts align with the predicted prompts.
  arXiv  Detail & Related papers  (2025-02-21T19:41:32Z)
• Who Writes What: Unveiling the Impact of Author Roles on AI-generated Text Detection [44.05134959039957]
  We investigate how sociolinguistic attributes-gender, CEFR proficiency, academic field, and language environment-impact state-of-the-art AI text detectors.<n>Our results reveal significant biases: CEFR proficiency and language environment consistently affected detector accuracy, while gender and academic field showed detector-dependent effects.<n>These findings highlight the crucial need for socially aware AI text detection to avoid unfairly penalizing specific demographic groups.
  arXiv  Detail & Related papers  (2025-02-18T07:49:31Z)
• Where are we in audio deepfake detection? A systematic analysis over generative and detection models [59.09338266364506]
  SONAR is a synthetic AI-Audio Detection Framework and Benchmark.<n>It provides a comprehensive evaluation for distinguishing cutting-edge AI-synthesized auditory content.<n>It is the first framework to uniformly benchmark AI-audio detection across both traditional and foundation model-based detection systems.
  arXiv  Detail & Related papers  (2024-10-06T01:03:42Z)
• On the Possibilities of AI-Generated Text Detection [76.55825911221434]
  We argue that as machine-generated text approximates human-like quality, the sample size needed for detection bounds increases. We test various state-of-the-art text generators, including GPT-2, GPT-3.5-Turbo, Llama, Llama-2-13B-Chat-HF, and Llama-2-70B-Chat-HF, against detectors, including oBERTa-Large/Base-Detector, GPTZero.
  arXiv  Detail & Related papers  (2023-04-10T17:47:39Z)
• AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses [66.49753193098356]
  We investigate the reason behind the surprising adversarial brittleness of scoring models. Our results indicate that autoscoring models, despite getting trained as "end-to-end" models, behave like bag-of-words models. We propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.
  arXiv  Detail & Related papers  (2021-09-24T03:49:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.