Beyond Perplexity: Character Distribution Signatures and the MDTA Benchmark for AI Text Detection

• URL: http://arxiv.org/abs/2605.01647v1
• Date: Sun, 03 May 2026 00:10:15 GMT
• Title: Beyond Perplexity: Character Distribution Signatures and the MDTA Benchmark for AI Text Detection
• Abstract summary: Training-free AI text detection methods primarily rely on model log-probabilities, achieving strong performance through approaches like Binoculars and DNA-DetectLLM.<n>We introduce an alternative detection signal based on character distribution signatures.<n>We provide theoretical foundations showing that AI models, trained on massive domain-balanced corpora, approximate global character patterns while humans exhibit domain-specialized distributions.
• Score: 0.34998703934432673
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Training-free AI text detection methods primarily rely on model log-probabilities, achieving strong performance through approaches like Binoculars and DNA-DetectLLM. However, these methods face a fundamental ceiling as models are optimized through RLHF to produce human-like probability distributions. We introduce an alternative detection signal based on character distribution signatures. We provide theoretical foundations showing that AI models, trained on massive domain-balanced corpora, approximate global character patterns while humans exhibit domain-specialized distributions, creating a "Wall of Separation" where human-AI divergence significantly exceeds AI-AI divergence. To enable systematic evaluation, we construct the Models-Domains-Temperatures-Adversarials (MDTA) benchmark comprising 642,274 prompt-aligned samples across 4 models, 5 domains, 3 temperature settings, and 3 adversarial strategies, substantially expanding the HC3 dataset with modern model responses, temperature variation, and adversarial augmentation. We introduce the Letter Distribution Score (LD-Score), demonstrating low correlation (r = 0.08-0.13) with perplexity methods. When integrated with DNA-DetectLLM, Binoculars and FastDetectGPT via a non-linear classifier, LD-Score yields consistent improvements in AUROC and F1, with particularly pronounced gains in specialized domains where vocabulary constraints amplify the detection signal. The MDTA dataset can be accessed at: https://huggingface.co/datasets/nsp909/MDTA.

Related papers

• A Unified Detection Framework for AI-Related Content and Artifacts [5.8904725273823475]
  We propose a unified detection framework based on Mahalanobis distance scores (MDS)<n>A key component of the proposed method is to accurately characterize the positive class--such as human-generated text, factual statements, unwatermarked text, or non-adversarial samples.
  arXiv  Detail & Related papers  (2026-07-08T15:21:07Z)
• Anatomically-conditioned Latent Diffusion Model for Data-Efficient Few-Shot Cross-Domain 3D Glioma MRI Synthesis [1.4312099612976772]
  Anatomically-conditioned Latent Diffusion Model (ALDM) is a novel framework for data-efficient, few-shot 3D MRI synthesis.<n> evaluated in an extreme few-shot setting with only 16 target images.
  arXiv  Detail & Related papers  (2026-06-24T04:42:42Z)
• OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generation [100.36917520403227]
  We present a unified text-conditioned diffusion framework that generates LiDAR scans in a shared range-image representation across eight representative domains.<n>In the absence of a public consolidated benchmark, we construct an 8-domain dataset by combining real-world scans with physically based weather simulation and systematic beam reduction.
  arXiv  Detail & Related papers  (2026-05-13T17:42:20Z)
• Feature-Augmented Transformers for Robust AI-Text Detection Across Domains and Generators [0.10923877073891443]
  We train transformer-based detectors on HC3 PLUS and calibrate a single decision threshold by maximising balanced accuracy on held-out validation.<n>We evaluate in-domain on HC3 PLUS, under cross-dataset transfer to the multi-domain, multi-generator M4 benchmark, and on the external AI-Text-Detection-Pile.<n>Our best model (DeBERTa-v3-base+FeatAttn) achieves 85.9% balanced accuracy on M4.
  arXiv  Detail & Related papers  (2026-05-05T16:52:26Z)
• Detecting the Machine: A Comprehensive Benchmark of AI-Generated Text Detectors Across Architectures, Domains, and Adversarial Conditions [0.0]
  Existing benchmarks evaluate a single detector on a single dataset under ideal conditions.<n>We present a benchmark evaluating diverse detection approaches across two corpora.<n>Results show that transformer models achieve near-perfect in-distribution performance but degrade under domain shift.
  arXiv  Detail & Related papers  (2026-03-18T09:27:27Z)
• Unsupervised domain adaptation for radioisotope identification in gamma spectroscopy [0.0]
  Unsupervised domain adaptation (UDA) can improve the ability of a model trained on synthetic data to generalize to a new testing domain.<n>In this study, we demonstrate that UDA can improve the ability of a model trained on synthetic data to generalize to a new testing domain.
  arXiv  Detail & Related papers  (2026-03-05T22:19:55Z)
• Multi-Hierarchical Feature Detection for Large Language Model Generated Text [2.5782420501870287]
  Multi-hierarchical feature integration for AI text detection is investigated.<n>We implement MHFD (Multi-Hierarchical Feature Detection), integrating semantic analysis, syntactic parsing, and statistical probability features through adaptive fusion.<n> Experimental results on multiple benchmark datasets demonstrate that the MHFD method achieves 89.7% accuracy in in-domain detection and maintains 84.2% stable performance in cross-domain detection, showing modest improvements of 0.4-2.6% over existing methods.
  arXiv  Detail & Related papers  (2025-09-23T09:55:42Z)
• Zero-Shot Image Anomaly Detection Using Generative Foundation Models [2.241618130319058]
  This research explores the use of score-based generative models as foundational tools for semantic anomaly detection.<n>By analyzing Stein score errors, we introduce a novel method for identifying anomalous samples without requiring re-training on each target dataset.<n>Our approach improves over state-of-the-art and relies on training a single model on one dataset -- CelebA -- which we find to be an effective base distribution.
  arXiv  Detail & Related papers  (2025-07-30T13:56:36Z)
• Topology-Aware Modeling for Unsupervised Simulation-to-Reality Point Cloud Recognition [63.55828203989405]
  We introduce a novel Topology-Aware Modeling (TAM) framework for Sim2Real UDA on object point clouds.<n>Our approach mitigates the domain gap by leveraging global spatial topology, characterized by low-level, high-frequency 3D structures.<n>We propose an advanced self-training strategy that combines cross-domain contrastive learning with self-training.
  arXiv  Detail & Related papers  (2025-06-26T11:53:59Z)
• Multi-Source and Test-Time Domain Adaptation on Multivariate Signals using Spatio-Temporal Monge Alignment [59.75420353684495]
  Machine learning applications on signals such as computer vision or biomedical data often face challenges due to the variability that exists across hardware devices or session recordings. In this work, we propose Spatio-Temporal Monge Alignment (STMA) to mitigate these variabilities. We show that STMA leads to significant and consistent performance gains between datasets acquired with very different settings.
  arXiv  Detail & Related papers  (2024-07-19T13:33:38Z)
• Forward-Forward Learning achieves Highly Selective Latent Representations for Out-of-Distribution Detection in Fully Spiking Neural Networks [6.7236795813629]
  Spiking Neural Networks (SNNs), inspired by biological systems, offer a promising avenue for overcoming limitations.<n>In this work, we explore the potential of the spiking Forward-Forward Algorithm (FFA) to address these challenges.<n>We propose a novel, gradient-free attribution method to detect features that drive a sample away from class distributions.
  arXiv  Detail & Related papers  (2024-07-19T08:08:17Z)
• Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution [67.9215891673174]
  We propose score entropy as a novel loss that naturally extends score matching to discrete spaces. We test our Score Entropy Discrete Diffusion models on standard language modeling tasks.
  arXiv  Detail & Related papers  (2023-10-25T17:59:12Z)
• MAUVE Scores for Generative Models: Theory and Practice [95.86006777961182]
  We present MAUVE, a family of comparison measures between pairs of distributions such as those encountered in the generative modeling of text or images. We find that MAUVE can quantify the gaps between the distributions of human-written text and those of modern neural language models. We demonstrate in the vision domain that MAUVE can identify known properties of generated images on par with or better than existing metrics.
  arXiv  Detail & Related papers  (2022-12-30T07:37:40Z)
• AvgOut: A Simple Output-Probability Measure to Eliminate Dull Responses [97.50616524350123]
  We build dialogue models that are dynamically aware of what utterances or tokens are dull without any feature-engineering. The first model, MinAvgOut, directly maximizes the diversity score through the output distributions of each batch. The second model, Label Fine-Tuning (LFT), prepends to the source sequence a label continuously scaled by the diversity score to control the diversity level. The third model, RL, adopts Reinforcement Learning and treats the diversity score as a reward signal.
  arXiv  Detail & Related papers  (2020-01-15T18:32:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.