Machine-Generated Text Localization

• URL: http://arxiv.org/abs/2402.11744v2
• Date: Mon, 10 Jun 2024 19:20:20 GMT
• Title: Machine-Generated Text Localization
• Authors: Zhongping Zhang, Wenda Qin, Bryan A. Plummer,
• Abstract summary: Prior work has primarily formulated MGT detection as a binary classification task over an entire document. This paper provides the first in-depth study of MGT that localizes the portions of a document that were machine generated. A gain of 4-13% mean Average Precision (mAP) over prior work demonstrates the effectiveness of approach on five diverse datasets.
• Score: 16.137882615106523
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine-Generated Text (MGT) detection aims to identify a piece of text as machine or human written. Prior work has primarily formulated MGT detection as a binary classification task over an entire document, with limited work exploring cases where only part of a document is machine generated. This paper provides the first in-depth study of MGT that localizes the portions of a document that were machine generated. Thus, if a bad actor were to change a key portion of a news article to spread misinformation, whole document MGT detection may fail since the vast majority is human written, but our approach can succeed due to its granular approach. A key challenge in our MGT localization task is that short spans of text, e.g., a single sentence, provides little information indicating if it is machine generated due to its short length. To address this, we leverage contextual information, where we predict whether multiple sentences are machine or human written at once. This enables our approach to identify changes in style or content to boost performance. A gain of 4-13% mean Average Precision (mAP) over prior work demonstrates the effectiveness of approach on five diverse datasets: GoodNews, VisualNews, WikiText, Essay, and WP. We release our implementation at https://github.com/Zhongping-Zhang/MGT_Localization.

Related papers

• GigaCheck: Detecting LLM-generated Content [72.27323884094953]
  In this work, we investigate the task of generated text detection by proposing the GigaCheck. Our research explores two approaches: (i) distinguishing human-written texts from LLM-generated ones, and (ii) detecting LLM-generated intervals in Human-Machine collaborative texts. Specifically, we use a fine-tuned general-purpose LLM in conjunction with a DETR-like detection model, adapted from computer vision, to localize artificially generated intervals within text.
  arXiv  Detail & Related papers  (2024-10-31T08:30:55Z)
• LLM-DetectAIve: a Tool for Fine-Grained Machine-Generated Text Detection [87.43727192273772]
  It is often hard to tell whether a piece of text was human-written or machine-generated. We present LLM-DetectAIve, designed for fine-grained detection. It supports four categories: (i) human-written, (ii) machine-generated, (iii) machine-written, then machine-humanized, and (iv) human-written, then machine-polished.
  arXiv  Detail & Related papers  (2024-08-08T07:43:17Z)
• M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection [69.41274756177336]
  Large Language Models (LLMs) have brought an unprecedented surge in machine-generated text (MGT) across diverse channels. This raises legitimate concerns about its potential misuse and societal implications. We introduce a new benchmark based on a multilingual, multi-domain, and multi-generator corpus of MGTs -- M4GT-Bench.
  arXiv  Detail & Related papers  (2024-02-17T02:50:33Z)
• M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection [69.29017069438228]
  Large language models (LLMs) have demonstrated remarkable capability to generate fluent responses to a wide variety of user queries. This has also raised concerns about the potential misuse of such texts in journalism, education, and academia. In this study, we strive to create automated systems that can detect machine-generated texts and pinpoint potential misuse.
  arXiv  Detail & Related papers  (2023-05-24T08:55:11Z)
• Smaller Language Models are Better Black-box Machine-Generated Text Detectors [56.36291277897995]
  Small and partially-trained models are better universal text detectors. We find that whether the detector and generator were trained on the same data is not critically important to the detection success. For instance, the OPT-125M model has an AUC of 0.81 in detecting ChatGPT generations, whereas a larger model from the GPT family, GPTJ-6B, has AUC of 0.45.
  arXiv  Detail & Related papers  (2023-05-17T00:09:08Z)
• SelfDocSeg: A Self-Supervised vision-based Approach towards Document Segmentation [15.953725529361874]
  Document layout analysis is a known problem to the documents research community. With growing internet connectivity to personal life, an enormous amount of documents had been available in the public domain. We address this challenge using self-supervision and unlike, the few existing self-supervised document segmentation approaches.
  arXiv  Detail & Related papers  (2023-05-01T12:47:55Z)
• CoCo: Coherence-Enhanced Machine-Generated Text Detection Under Data Limitation With Contrastive Learning [14.637303913878435]
  We present a coherence-based contrastive learning model named CoCo to detect the possible MGT under low-resource scenario. To exploit the linguistic feature, we encode coherence information in form of graph into text representation. Experiment results on two public datasets and two self-constructed datasets prove our approach outperforms the state-of-art methods significantly.
  arXiv  Detail & Related papers  (2022-12-20T15:26:19Z)
• Unsupervised and Distributional Detection of Machine-Generated Text [1.552214657968262]
  The power of natural language generation models has provoked a flurry of interest in automatic methods to detect if a piece of text is human or machine-authored. We propose a method to detect those machine-generated documents leveraging repeated higher-order n-grams. Our experiments show that leveraging that signal allows us to rank suspicious documents accurately.
  arXiv  Detail & Related papers  (2021-11-04T14:07:46Z)
• GLEAKE: Global and Local Embedding Automatic Keyphrase Extraction [1.0681288493631977]
  We introduce Global and Local Embedding Automatic Keyphrase Extractor (GLEAKE) for the task of automatic keyphrase extraction. GLEAKE uses single and multi-word embedding techniques to explore the syntactic and semantic aspects of the candidate phrases. It refines the most significant phrases as a final set of keyphrases.
  arXiv  Detail & Related papers  (2020-05-19T20:24:02Z)
• Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension [59.80926970481975]
  We study machine reading comprehension (MRC) on long texts. A model takes as inputs a lengthy document and a question and then extracts a text span from the document as an answer. We propose to let a model learn to chunk in a more flexible way via reinforcement learning.
  arXiv  Detail & Related papers  (2020-05-16T18:08:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.