Style as a Confound: False Positives in AI Detection of Non-Native Academic Writing

• URL: http://arxiv.org/abs/2608.26710v1
• Date: Thu, 27 Aug 2026 07:05:31 GMT
• Title: Style as a Confound: False Positives in AI Detection of Non-Native Academic Writing
• Abstract summary: We examined 135,389 document pairs from a professional English editing service.<n>For the 13 AI text detectors, FPRs for human-written texts varied widely, from 0.0% to 100.0%.<n>The findings identify professional editing style as a key confounding variable in AI detector outputs.
• Score: 3.9444299467643025
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI text detectors are increasingly employed in academic settings, but it remains unclear whether their outputs reflect AI authorship itself or broader linguistic features associated with polished academic English. Previous studies have reported high false-positive rates (FPRs) for non-native English writing, but population-level comparisons confound authorship with differences in topic, domain, and writing style. Professional editing provides a useful setting for examining this issue because it changes the linguistic form of manuscripts while preserving authorship and content. We examined 135,389 document pairs from a professional English editing service (2018-2025), comprising non-native manuscripts and their native-edited versions, to assess how editing affects detector responses controlling for content and authorship. For the 13 AI text detectors, FPRs for human-written texts varied widely, from 0.0% to 100.0%. Responses varied across detectors: the same edits increased AI scores in some detectors but decreased them in others. Notably, score changes correlated with the extent of editing. The findings identify professional editing style as a key confounding variable in AI detector outputs, rather than establishing a full separation of text origin from linguistic style, raising concerns about fairness and reliability in academic settings.

Related papers

• PaperMentor: A Human-Centered Multi-Agent Writing Tutor for AI Research Papers on Overleaf [51.79108552674703]
  We present PaperMentor, a human-centered writing assistant system that delivers actionable suggestions as Overleaf-native inline comments.<n>PaperMentor integrates an expert skill library carefully curated from established researchers' writing advice with 12 specialized agents covering different aspects of paper writing.<n>In a user study, 90.6% of the generated comments were rated actionable and 67.5% were rated valid, significantly outperforming a GPT-5.2 baseline uswithout the skill library.
  arXiv  Detail & Related papers  (2026-06-07T22:17:15Z)
• Operation-Guided Progressive Human-to-AI Text Transformation Benchmark for Multi-Granularity AI-Text Detection [49.41415417307568]
  OpAI-Bench is an operation-guided benchmark for studying progressive human-to-AI text transformation.<n>We show that AI-text detectability is governed not only by the proportion of AI-edited content, but also by edit operation, domain, and cumulative revision history.
  arXiv  Detail & Related papers  (2026-06-04T17:58:05Z)
• The Impact of Editorial Intervention on Detecting Native Language Traces [1.6210510806456278]
  Native Language Identification (NLI) is the task of determining an author's native language (L1) from their non-native writings.<n>With the advent of human-AI co-authorship, non-native texts are routinely corrected and rewritten by large language models.<n>We investigate the robustness of L1 traces across increasing degrees of editorial intervention.
  arXiv  Detail & Related papers  (2026-05-11T08:58:08Z)
• Explainable Disentangled Representation Learning for Generalizable Authorship Attribution in the Era of Generative AI [70.05655131235999]
  Explainable Authorship Variational Autoencoder (EAVAE) is a novel framework that explicitly disentangles style from content.<n>Disentanglement is enforced through a novel discriminator.<n>EAVAE excels in few-shot learning over the M4 dataset for AI-generated text detection.
  arXiv  Detail & Related papers  (2026-04-23T05:34:20Z)
• Can professional translators identify machine-generated text? [0.0]
  This study investigates whether professional translators can reliably identify short stories generated in Italian by artificial intelligence (AI) without prior specialized training.<n>Sixty-nine translators took part in an in-person experiment, where they assessed three anonymized short stories.<n>Low burstiness and narrative contradiction emerged as the most reliable indicators of synthetic authorship.
  arXiv  Detail & Related papers  (2026-01-22T10:25:52Z)
• EditLens: Quantifying the Extent of AI Editing in Text [23.457378805409714]
  We show that AI-edited text is distinguishable from human-written and AI-generated text.<n>We train a regression model that predicts the amount of AI editing present within a text.<n>Not only do we show that AI-edited text can be detected, but also that the degree of change made by AI to human writing can be detected.
  arXiv  Detail & Related papers  (2025-10-03T16:27:48Z)
• Group-Adaptive Threshold Optimization for Robust AI-Generated Text Detection [58.419940585826744]
  We introduce FairOPT, an algorithm for group-specific threshold optimization for probabilistic AI-text detectors.<n>We partitioned data into subgroups based on attributes (e.g., text length and writing style) and implemented FairOPT to learn decision thresholds for each group to reduce discrepancy.<n>Our framework paves the way for more robust classification in AI-generated content detection via post-processing.
  arXiv  Detail & Related papers  (2025-02-06T21:58:48Z)
• Detecting the Undetectable: Assessing the Efficacy of Current Spoof Detection Methods Against Seamless Speech Edits [82.8859060022651]
  We introduce the Speech INfilling Edit (SINE) dataset, created with Voicebox.<n>Subjective evaluations confirm that speech edited using this novel technique is more challenging to detect than conventional cut-and-paste methods.<n>Despite human difficulty, experimental results demonstrate that self-supervised-based detectors can achieve remarkable performance in detection, localization, and generalization.
  arXiv  Detail & Related papers  (2025-01-07T14:17:47Z)
• Syntactic Language Change in English and German: Metrics, Parsers, and Convergences [56.47832275431858]
  The current paper looks at diachronic trends in syntactic language change in both English and German, using corpora of parliamentary debates from the last c. 160 years. We base our observations on five dependencys, including the widely used Stanford Core as well as 4 newer alternatives. We show that changes in syntactic measures seem to be more frequent at the tails of sentence length distributions.
  arXiv  Detail & Related papers  (2024-02-18T11:46:16Z)
• ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews [36.76200047558003]
  We introduce the task of automatically revising scientific papers based on peer feedback. We provide labels linking each reviewer comment to the specific paper edits made by the author in response. In experiments with 10 models covering the state of the art, we find that they struggle even to identify which edits correspond to a comment.
  arXiv  Detail & Related papers  (2023-06-21T22:00:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.