Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods

• URL: http://arxiv.org/abs/2210.07321v4
• Date: Mon, 8 May 2023 01:56:38 GMT
• Title: Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods
• Authors: Evan Crothers, Nathalie Japkowicz, Herna Viktor
• Abstract summary: This survey places machine generated text within its cybersecurity and social context. It includes an analysis of threat models posed by contemporary NLG systems. It provides guidance for future work addressing the most critical threat models.
• Score: 6.978441815839558
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine generated text is increasingly difficult to distinguish from human authored text. Powerful open-source models are freely available, and user-friendly tools that democratize access to generative models are proliferating. ChatGPT, which was released shortly after the first edition of this survey, epitomizes these trends. The great potential of state-of-the-art natural language generation (NLG) systems is tempered by the multitude of avenues for abuse. Detection of machine generated text is a key countermeasure for reducing abuse of NLG models, with significant technical challenges and numerous open problems. We provide a survey that includes both 1) an extensive analysis of threat models posed by contemporary NLG systems, and 2) the most complete review of machine generated text detection methods to date. This survey places machine generated text within its cybersecurity and social context, and provides strong guidance for future work addressing the most critical threat models, and ensuring detection systems themselves demonstrate trustworthiness through fairness, robustness, and accountability.

Related papers

• In-Context Experience Replay Facilitates Safety Red-Teaming of Text-to-Image Diffusion Models [97.82118821263825]
  Text-to-image (T2I) models have shown remarkable progress, but their potential to generate harmful content remains a critical concern in the ML community. We propose ICER, a novel red-teaming framework that generates interpretable and semantic meaningful problematic prompts. Our work provides crucial insights for developing more robust safety mechanisms in T2I systems.
  arXiv  Detail & Related papers  (2024-11-25T04:17:24Z)
• Detecting Machine-Generated Long-Form Content with Latent-Space Variables [54.07946647012579]
  Existing zero-shot detectors primarily focus on token-level distributions, which are vulnerable to real-world domain shifts. We propose a more robust method that incorporates abstract elements, such as event transitions, as key deciding factors to detect machine versus human texts.
  arXiv  Detail & Related papers  (2024-10-04T18:42:09Z)
• Humanizing Machine-Generated Content: Evading AI-Text Detection through Adversarial Attack [24.954755569786396]
  We propose a framework for a broader class of adversarial attacks, designed to perform minor perturbations in machine-generated content to evade detection. We consider two attack settings: white-box and black-box, and employ adversarial learning in dynamic scenarios to assess the potential enhancement of the current detection model's robustness. The empirical results reveal that the current detection models can be compromised in as little as 10 seconds, leading to the misclassification of machine-generated text as human-written content.
  arXiv  Detail & Related papers  (2024-04-02T12:49:22Z)
• Evaluating Robustness of Generative Search Engine on Adversarial Factual Questions [89.35345649303451]
  Generative search engines have the potential to transform how people seek information online. But generated responses from existing large language models (LLMs)-backed generative search engines may not always be accurate. Retrieval-augmented generation exacerbates safety concerns, since adversaries may successfully evade the entire system.
  arXiv  Detail & Related papers  (2024-02-25T11:22:19Z)
• Detection of Machine-Generated Text: Literature Survey [0.0]
  This literature survey aims to compile and synthesize accomplishments and developments in the field of machine-generated text. It also gives an overview of machine-generated text trends and explores the larger societal implications.
  arXiv  Detail & Related papers  (2024-01-02T01:44:15Z)
• Towards Possibilities & Impossibilities of AI-generated Text Detection: A Survey [97.33926242130732]
  Large Language Models (LLMs) have revolutionized the domain of natural language processing (NLP) with remarkable capabilities of generating human-like text responses. Despite these advancements, several works in the existing literature have raised serious concerns about the potential misuse of LLMs. To address these concerns, a consensus among the research community is to develop algorithmic solutions to detect AI-generated text.
  arXiv  Detail & Related papers  (2023-10-23T18:11:32Z)
• FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios [87.12753459582116]
  A wider range of tasks now face an increasing risk of containing factual errors when handled by generative models. We propose FacTool, a task and domain agnostic framework for detecting factual errors of texts generated by large language models.
  arXiv  Detail & Related papers  (2023-07-25T14:20:51Z)
• G3Detector: General GPT-Generated Text Detector [26.47122201110071]
  We introduce an unpretentious yet potent detection approach proficient in identifying synthetic text across a wide array of fields. Our detector demonstrates outstanding performance uniformly across various model architectures and decoding strategies. It also possesses the capability to identify text generated utilizing a potent detection-evasion technique.
  arXiv  Detail & Related papers  (2023-05-22T03:35:00Z)
• On the Possibilities of AI-Generated Text Detection [76.55825911221434]
  We argue that as machine-generated text approximates human-like quality, the sample size needed for detection bounds increases. We test various state-of-the-art text generators, including GPT-2, GPT-3.5-Turbo, Llama, Llama-2-13B-Chat-HF, and Llama-2-70B-Chat-HF, against detectors, including oBERTa-Large/Base-Detector, GPTZero.
  arXiv  Detail & Related papers  (2023-04-10T17:47:39Z)
• A Comprehensive Survey of Natural Language Generation Advances from the Perspective of Digital Deception [1.557442325082254]
  We provide an overview of the field of natural language generators (NLG) We outline a proposed high-level taxonomy of the central concepts that constitute NLG. We discuss the broader challenges of NLG, including the risks of bias that are often exhibited by existing text generation systems.
  arXiv  Detail & Related papers  (2022-08-11T11:27:38Z)
• RoFT: A Tool for Evaluating Human Detection of Machine-Generated Text [25.80571756447762]
  We present Real or Fake Text (RoFT), a website that invites users to try their hand at detecting machine-generated text. We show preliminary results of using RoFT to evaluate detection of machine-generated news articles.
  arXiv  Detail & Related papers  (2020-10-06T22:47:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.