Detecting AI-Generated Texts in Cross-Domains

• URL: http://arxiv.org/abs/2410.13966v1
• Date: Thu, 17 Oct 2024 18:43:30 GMT
• Title: Detecting AI-Generated Texts in Cross-Domains
• Authors: You Zhou, Jie Wang,
• Abstract summary: We train a ranking classifier called RoBERTa-Ranker as a baseline model. We then present a method to fine-tune RoBERTa-Ranker that requires only a small amount of labeled data in a new domain. Experiments show that this fine-tuned domain-aware model outperforms the popular DetectGPT and GPTZero.
• Score: 3.2245324254437846
• License:
• Abstract: Existing tools to detect text generated by a large language model (LLM) have met with certain success, but their performance can drop when dealing with texts in new domains. To tackle this issue, we train a ranking classifier called RoBERTa-Ranker, a modified version of RoBERTa, as a baseline model using a dataset we constructed that includes a wider variety of texts written by humans and generated by various LLMs. We then present a method to fine-tune RoBERTa-Ranker that requires only a small amount of labeled data in a new domain. Experiments show that this fine-tuned domain-aware model outperforms the popular DetectGPT and GPTZero on both in-domain and cross-domain texts, where AI-generated texts may either be in a different domain or generated by a different LLM not used to generate the training datasets. This approach makes it feasible and economical to build a single system to detect AI-generated texts across various domains.

Related papers

• GigaCheck: Detecting LLM-generated Content [72.27323884094953]
  In this work, we investigate the task of generated text detection by proposing the GigaCheck. Our research explores two approaches: (i) distinguishing human-written texts from LLM-generated ones, and (ii) detecting LLM-generated intervals in Human-Machine collaborative texts. Specifically, we use a fine-tuned general-purpose LLM in conjunction with a DETR-like detection model, adapted from computer vision, to localize artificially generated intervals within text.
  arXiv  Detail & Related papers  (2024-10-31T08:30:55Z)
• Robust AI-Generated Text Detection by Restricted Embeddings [6.745955674138081]
  We focus on robustness of detectors of AI-generated text, namely their ability to transfer to unseen generators or semantic domains. We show that clearing out harmful linear subspaces helps to train a robust classifier, ignoring domain-specific spurious features. Our best approaches for head-wise and coordinate-based subspace removal increase the mean out-of-distribution (OOD) classification score by up to 9% and 14% in particular.
  arXiv  Detail & Related papers  (2024-10-10T16:58:42Z)
• Detecting Machine-Generated Long-Form Content with Latent-Space Variables [54.07946647012579]
  Existing zero-shot detectors primarily focus on token-level distributions, which are vulnerable to real-world domain shifts. We propose a more robust method that incorporates abstract elements, such as event transitions, as key deciding factors to detect machine versus human texts.
  arXiv  Detail & Related papers  (2024-10-04T18:42:09Z)
• Spotting AI's Touch: Identifying LLM-Paraphrased Spans in Text [61.22649031769564]
  We propose a novel framework, paraphrased text span detection (PTD) PTD aims to identify paraphrased text spans within a text. We construct a dedicated dataset, PASTED, for paraphrased text span detection.
  arXiv  Detail & Related papers  (2024-05-21T11:22:27Z)
• EAGLE: A Domain Generalization Framework for AI-generated Text Detection [15.254775341371364]
  We propose a domain generalization framework for the detection of AI-generated text from unseen target generators. We demonstrate how our framework effectively achieves impressive performance in detecting text generated by unseen target generators.
  arXiv  Detail & Related papers  (2024-03-23T02:44:20Z)
• A Simple yet Efficient Ensemble Approach for AI-generated Text Detection [0.5840089113969194]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in generating text that closely resembles human writing. It is essential to build automated approaches capable of distinguishing between artificially generated text and human-authored text. We propose a simple yet efficient solution by ensembling predictions from multiple constituent LLMs.
  arXiv  Detail & Related papers  (2023-11-06T13:11:02Z)
• Corpus Synthesis for Zero-shot ASR domain Adaptation using Large Language Models [19.726699481313194]
  We propose a new strategy for adapting ASR models to new target domains without any text or speech from those domains. Experiments on the SLURP dataset show that the proposed method achieves an average relative word error rate improvement of $28%$ on unseen target domains.
  arXiv  Detail & Related papers  (2023-09-18T15:43:08Z)
• MAGE: Machine-generated Text Detection in the Wild [82.70561073277801]
  Large language models (LLMs) have achieved human-level text generation, emphasizing the need for effective AI-generated text detection. We build a comprehensive testbed by gathering texts from diverse human writings and texts generated by different LLMs. Despite challenges, the top-performing detector can identify 86.54% out-of-domain texts generated by a new LLM, indicating the feasibility for application scenarios.
  arXiv  Detail & Related papers  (2023-05-22T17:13:29Z)
• Bidirectional Generative Framework for Cross-domain Aspect-based Sentiment Analysis [68.742820522137]
  Cross-domain aspect-based sentiment analysis (ABSA) aims to perform various fine-grained sentiment analysis tasks on a target domain by transferring knowledge from a source domain. We propose a unified bidirectional generative framework to tackle various cross-domain ABSA tasks. Our framework trains a generative model in both text-to-label and label-to-text directions.
  arXiv  Detail & Related papers  (2023-05-16T15:02:23Z)
• Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation [49.89831914386982]
  We propose a unified pre-trained language model (PLM) for all forms of text, including unstructured text, semi-structured text, and well-structured text. Our approach outperforms the pre-training of plain text using only 1/4 of the data.
  arXiv  Detail & Related papers  (2021-09-02T16:05:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.