Operation-Guided Progressive Human-to-AI Text Transformation Benchmark for Multi-Granularity AI-Text Detection

• URL: http://arxiv.org/abs/2606.06481v1
• Date: Thu, 04 Jun 2026 17:58:05 GMT
• Title: Operation-Guided Progressive Human-to-AI Text Transformation Benchmark for Multi-Granularity AI-Text Detection
• Abstract summary: OpAI-Bench is an operation-guided benchmark for studying progressive human-to-AI text transformation.<n>We show that AI-text detectability is governed not only by the proportion of AI-edited content, but also by edit operation, domain, and cumulative revision history.
• Score: 49.41415417307568
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As AI writing assistants become increasingly integrated into real-world drafting and revision workflows, many documents are no longer purely human-written or AI-generated, but instead result from progressive human-AI co-editing. However, existing AI-text detection benchmarks largely focus on final outputs and provide limited understanding of how AI authorship signals emerge, accumulate, or disappear throughout the revision process. We introduce OpAI-Bench, an operation-guided benchmark for studying progressive human-to-AI text transformation across document, sentence, token, and span granularities. Starting from human-written documents, OpAI-Bench constructs nine sequentially revised versions for each sample under predefined AI coverage levels and five representative AI edit operations, covering four domains while preserving complete authorship provenance at multiple granularities. The benchmark supports comprehensive evaluation with 8 document-level detectors, 7 sentence-level detectors, and 2 fine-grained token/span-level detectors. Experiments reveal that AI-text detectability is governed not only by the proportion of AI-edited content, but also by edit operation, domain, and cumulative revision history. Interestingly, we notice that mixed-authorship intermediate versions are often harder to detect than both fully human and heavily AI-edited endpoints, exposing non-monotonic detection patterns missed by existing benchmarks. OpAI-Bench provides a controlled testbed for analyzing whether, when, and how AI-assisted writing becomes detectable under realistic progressive editing scenarios. Our code and benchmark are available at https://github.com/VILA-Lab/OpAI-Bench.

Related papers

• 'Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions [50.1119556361129]
  We systematically define various notions of AI-generated text and their characteristics.<n>AITDNA is a new benchmark of human-machine co-constructed texts.<n>We benchmark various machine-generated text detectors and find that they often only perform well for specific notions but not as broad detectors.
  arXiv  Detail & Related papers  (2026-06-03T14:05:14Z)
• Explainable Disentangled Representation Learning for Generalizable Authorship Attribution in the Era of Generative AI [70.05655131235999]
  Explainable Authorship Variational Autoencoder (EAVAE) is a novel framework that explicitly disentangles style from content.<n>Disentanglement is enforced through a novel discriminator.<n>EAVAE excels in few-shot learning over the M4 dataset for AI-generated text detection.
  arXiv  Detail & Related papers  (2026-04-23T05:34:20Z)
• EditLens: Quantifying the Extent of AI Editing in Text [23.457378805409714]
  We show that AI-edited text is distinguishable from human-written and AI-generated text.<n>We train a regression model that predicts the amount of AI editing present within a text.<n>Not only do we show that AI-edited text can be detected, but also that the degree of change made by AI to human writing can be detected.
  arXiv  Detail & Related papers  (2025-10-03T16:27:48Z)
• Fine-Grained Detection of AI-Generated Text Using Sentence-Level Segmentation [3.088244520495001]
  A sentence-level sequence labeling model proposed to detect transitions between human- and AI-generated text.<n>Our model combines the state-of-the-art pre-trained Transformer models, incorporating Neural Networks (NN) and Conditional Random Fields (CRFs)<n>The evaluation is performed on two publicly available benchmark datasets containing collaborative human and AI-generated texts.
  arXiv  Detail & Related papers  (2025-09-22T14:22:55Z)
• DNA-DetectLLM: Unveiling AI-Generated Text via a DNA-Inspired Mutation-Repair Paradigm [17.258462909671525]
  We introduce DNA-DetectLLM, a zero-shot detection method for distinguishing AI-generated and human-written text.<n>DNA-DetectLLM achieves relative improvements of 5.55% in AUROC and 2.08% in F1 score across multiple public benchmark datasets.
  arXiv  Detail & Related papers  (2025-09-19T03:08:13Z)
• HACo-Det: A Study Towards Fine-Grained Machine-Generated Text Detection under Human-AI Coauthoring [14.887491317701997]
  This paper explores the possibility of fine-grained MGT detection under human-AI coauthoring.<n>We suggest fine-grained detectors can pave pathways toward coauthored text detection with a numeric AI ratio.<n> Empirical results show that metric-based methods struggle to conduct fine-grained detection with a 0.462 average F1 score.
  arXiv  Detail & Related papers  (2025-06-03T14:52:44Z)
• Is Your Paper Being Reviewed by an LLM? Benchmarking AI Text Detection in Peer Review [6.20631177269082]
  A new risk to the peer review process is that negligent reviewers will rely on large language models (LLMs) to review a paper.<n>We introduce a comprehensive dataset containing a total of 788,984 AI-written peer reviews paired with corresponding human reviews.<n>We use this new resource to evaluate the ability of 18 existing AI text detection algorithms to distinguish between peer reviews fully written by humans and different state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2025-02-26T23:04:05Z)
• Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing [55.2480439325792]
  This study systematically evaluations twelve state-of-the-art AI-text detectors using our AI-Polished-Text Evaluation dataset.<n>Our findings reveal that detectors frequently flag even minimally polished text as AI-generated, struggle to differentiate between degrees of AI involvement, and exhibit biases against older and smaller models.
  arXiv  Detail & Related papers  (2025-02-21T18:45:37Z)
• Group-Adaptive Threshold Optimization for Robust AI-Generated Text Detection [58.419940585826744]
  We introduce FairOPT, an algorithm for group-specific threshold optimization for probabilistic AI-text detectors.<n>We partitioned data into subgroups based on attributes (e.g., text length and writing style) and implemented FairOPT to learn decision thresholds for each group to reduce discrepancy.<n>Our framework paves the way for more robust classification in AI-generated content detection via post-processing.
  arXiv  Detail & Related papers  (2025-02-06T21:58:48Z)
• On the Possibilities of AI-Generated Text Detection [76.55825911221434]
  We argue that as machine-generated text approximates human-like quality, the sample size needed for detection bounds increases. We test various state-of-the-art text generators, including GPT-2, GPT-3.5-Turbo, Llama, Llama-2-13B-Chat-HF, and Llama-2-70B-Chat-HF, against detectors, including oBERTa-Large/Base-Detector, GPTZero.
  arXiv  Detail & Related papers  (2023-04-10T17:47:39Z)
• Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense [56.077252790310176]
  We present a paraphrase generation model (DIPPER) that can paraphrase paragraphs, condition on surrounding context, and control lexical diversity and content reordering. Using DIPPER to paraphrase text generated by three large language models (including GPT3.5-davinci-003) successfully evades several detectors, including watermarking. We introduce a simple defense that relies on retrieving semantically-similar generations and must be maintained by a language model API provider.
  arXiv  Detail & Related papers  (2023-03-23T16:29:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.