AI Detectors Fail Diverse Student Populations: A Mathematical Framing of Structural Detection Limits

• URL: http://arxiv.org/abs/2603.20254v1
• Date: Wed, 11 Mar 2026 06:38:31 GMT
• Title: AI Detectors Fail Diverse Student Populations: A Mathematical Framing of Structural Detection Limits
• Abstract summary: "Black box" AI text detectors produce high false positive rates with disproportionate errors against certain student populations.<n>We argue that detection scores should not serve as sole evidence in misconduct proceedings.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Student experiences and empirical studies report that "black box" AI text detectors produce high false positive rates with disproportionate errors against certain student populations, yet typically theoretical analyses model detection as a test between two known distributions for human and AI prose. This framing omits the structural feature of university assessment whereby an assessor generally does not know the individual student's writing distribution, making the null hypothesis composite. Standard application of the variational characterisation of total variation distance to this composite null shows trade-off bounds that any text-only, one-shot detector with useful power must produce false accusations at a rate governed by the distributional overlap between student writing and AI output. This is a constraint arising from population diversity that is logically independent of AI model quality and cannot be overcome by better detector engineering or technology. A subgroup mixture bound connects these quantities to observable demographic groups, providing a theoretical basis for the disparate impact patterns documented empirically. We propose suggestions to improve policy and practice, and argue that detection scores should not serve as sole evidence in misconduct proceedings.

Related papers

• Policies Permitting LLM Use for Polishing Peer Reviews Are Currently Not Enforceable [17.525659482226875]
  A number of scientific conferences and journals have recently enacted policies that prohibit LLM usage by peer reviewers.<n>We assemble a dataset of peer reviews simulating multiple levels of human-AI collaboration, and evaluate five state-of-the-art detectors.<n>Our analysis shows that all detectors misclassify a non-trivial fraction of LLM-polished reviews as AI-generated, risking false accusations of academic misconduct.
  arXiv  Detail & Related papers  (2026-03-20T19:28:34Z)
• BAID: A Benchmark for Bias Assessment of AI Detectors [9.156813547624923]
  We propose BAID, a comprehensive evaluation framework for AI detectors across various types of biases.<n>We introduce over 200k samples spanning 7 major categories: demographics, age, educational grade level, dialect, formality, political leaning, and topic.<n>We find consistent disparities in detection performance, particularly low recall rates for texts from underrepresented groups.
  arXiv  Detail & Related papers  (2025-12-12T12:01:42Z)
• Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection [71.59834293521074]
  We develop a framework to distinguish between human-authored and machine-generated text.<n>Our method achieves 98.3% AUROC and AUPR with only 8.9% FPR95 on DeepFake dataset.<n>Code, pretrained weights, and demo will be released.
  arXiv  Detail & Related papers  (2025-10-07T08:14:45Z)
• Fair Deepfake Detectors Can Generalize [51.21167546843708]
  We show that controlling for confounders (data distribution and model capacity) enables improved generalization via fairness interventions.<n>Motivated by this insight, we propose Demographic Attribute-insensitive Intervention Detection (DAID), a plug-and-play framework composed of: i) Demographic-aware data rebalancing, which employs inverse-propensity weighting and subgroup-wise feature normalization to neutralize distributional biases; and ii) Demographic-agnostic feature aggregation, which uses a novel alignment loss to suppress sensitive-attribute signals.<n>DAID consistently achieves superior performance in both fairness and generalization compared to several state-of-the-art
  arXiv  Detail & Related papers  (2025-07-03T14:10:02Z)
• Consistency-based Abductive Reasoning over Perceptual Errors of Multiple Pre-trained Models in Novel Environments [5.5855749614100825]
  This paper addresses the hypothesis that leveraging multiple pre-trained models can mitigate this recall reduction.<n>We formulate the challenge of identifying and managing conflicting predictions from various models as a consistency-based abduction problem.<n>Our results validate the use of consistency-based abduction as an effective mechanism to robustly integrate knowledge from multiple imperfect models in challenging, novel scenarios.
  arXiv  Detail & Related papers  (2025-05-25T23:17:47Z)
• Who Writes What: Unveiling the Impact of Author Roles on AI-generated Text Detection [44.05134959039957]
  We investigate how sociolinguistic attributes-gender, CEFR proficiency, academic field, and language environment-impact state-of-the-art AI text detectors.<n>Our results reveal significant biases: CEFR proficiency and language environment consistently affected detector accuracy, while gender and academic field showed detector-dependent effects.<n>These findings highlight the crucial need for socially aware AI text detection to avoid unfairly penalizing specific demographic groups.
  arXiv  Detail & Related papers  (2025-02-18T07:49:31Z)
• Group-Adaptive Threshold Optimization for Robust AI-Generated Text Detection [58.419940585826744]
  We introduce FairOPT, an algorithm for group-specific threshold optimization for probabilistic AI-text detectors.<n>We partitioned data into subgroups based on attributes (e.g., text length and writing style) and implemented FairOPT to learn decision thresholds for each group to reduce discrepancy.<n>Our framework paves the way for more robust classification in AI-generated content detection via post-processing.
  arXiv  Detail & Related papers  (2025-02-06T21:58:48Z)
• Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval [139.21955930418815]
  Cross-modal Retrieval methods build similarity relations between vision and language modalities by jointly learning a common representation space. However, the predictions are often unreliable due to the Aleatoric uncertainty, which is induced by low-quality data, e.g., corrupt images, fast-paced videos, and non-detailed texts. We propose a novel Prototype-based Aleatoric Uncertainty Quantification (PAU) framework to provide trustworthy predictions by quantifying the uncertainty arisen from the inherent data ambiguity.
  arXiv  Detail & Related papers  (2023-09-29T09:41:19Z)
• Assessing Student Errors in Experimentation Using Artificial Intelligence and Large Language Models: A Comparative Study with Human Raters [9.899633398596672]
  We investigate the potential of Large Language Models (LLMs) for automatically identifying student errors. An AI system based on the GPT-3.5 and GPT-4 series was developed and tested against human raters. Our results indicate varying levels of accuracy in error detection between the AI system and human raters.
  arXiv  Detail & Related papers  (2023-08-11T12:03:12Z)
• On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection [55.73320979733527]
  We propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs. Experimental results show that our method achieves competitive detection performance on various text classification tasks.
  arXiv  Detail & Related papers  (2023-06-27T02:54:07Z)
• Neural Causal Models for Counterfactual Identification and Estimation [62.30444687707919]
  We study the evaluation of counterfactual statements through neural models. First, we show that neural causal models (NCMs) are expressive enough. Second, we develop an algorithm for simultaneously identifying and estimating counterfactual distributions.
  arXiv  Detail & Related papers  (2022-09-30T18:29:09Z)
• Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
  We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers. We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model. Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
  arXiv  Detail & Related papers  (2020-06-22T21:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.