Ensemble Watermarks for Large Language Models

• URL: http://arxiv.org/abs/2411.19563v1
• Date: Fri, 29 Nov 2024 09:18:32 GMT
• Title: Ensemble Watermarks for Large Language Models
• Authors: Georg Niess, Roman Kern,
• Abstract summary: Acrostica and sensorimotor norms are combined with the established red-green watermark to achieve a 98% detection rate.<n>The evaluation of all feature combinations reveals that the ensemble of all three consistently has the highest detection rate.<n>This method is particularly of interest to facilitate accountability and prevent societal harm.
• Score: 1.89915151018241
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid advancement of large language models (LLMs) has made it increasingly difficult to distinguish between text written by humans and machines. While watermarks already exist for LLMs, they often lack flexibility, and struggle with attacks such as paraphrasing. To address these issues, we propose a multi-feature method for generating watermarks that combines multiple distinct watermark features into an ensemble watermark. Concretely, we combine acrostica and sensorimotor norms with the established red-green watermark to achieve a 98% detection rate. After a paraphrasing attack the performance remains high with 95% detection rate. The red-green feature alone as baseline achieves a detection rate of 49%. The evaluation of all feature combinations reveals that the ensemble of all three consistently has the highest detection rate across several LLMs and watermark strength settings. Due to the flexibility of combining features in the ensemble, various requirements and trade-offs can be addressed. Additionally, for all ensemble configurations the same detection function can be used without adaptations. This method is particularly of interest to facilitate accountability and prevent societal harm.

Related papers

• Entropy-Guided Watermarking for LLMs: A Test-Time Framework for Robust and Traceable Text Generation [58.85645136534301]
  Existing watermarking schemes for sampled text often face trade-offs between maintaining text quality and ensuring robust detection against various attacks. We propose a novel watermarking scheme that improves both detectability and text quality by introducing a cumulative watermark entropy threshold.
  arXiv  Detail & Related papers  (2025-04-16T14:16:38Z)
• DERMARK: A Dynamic, Efficient and Robust Multi-bit Watermark for Large Language Models [18.023143082876015]
  We propose DERMARK, a dynamic, efficient, and robust multi-bit watermarking method. DERMARK divides the text into segments of varying lengths for each bit embedding, adaptively matching the text's capacity. It achieves this with negligible overhead and robust performance against text editing by minimizing watermark extraction loss.
  arXiv  Detail & Related papers  (2025-02-04T11:23:49Z)
• Signal Watermark on Large Language Models [28.711745671275477]
  We propose a watermarking method embedding a specific watermark into the text during its generation by Large Language Models (LLMs) This technique not only ensures the watermark's invisibility to humans but also maintains the quality and grammatical integrity of model-generated text. Our method has been empirically validated across multiple LLMs, consistently maintaining high detection accuracy.
  arXiv  Detail & Related papers  (2024-10-09T04:49:03Z)
• Can Watermarked LLMs be Identified by Users via Crafted Prompts? [55.460327393792156]
  This work is the first to investigate the imperceptibility of watermarked Large Language Models (LLMs) We design an identification algorithm called Water-Probe that detects watermarks through well-designed prompts. Experiments show that almost all mainstream watermarking algorithms are easily identified with our well-designed prompts.
  arXiv  Detail & Related papers  (2024-10-04T06:01:27Z)
• Less is More: Sparse Watermarking in LLMs with Enhanced Text Quality [27.592486717044455]
  We present a novel type of watermark, Sparse Watermark, which aims to mitigate this trade-off by applying watermarks to a small subset of generated tokens distributed across the text. Our experimental results demonstrate that the proposed watermarking scheme achieves high detectability while generating text that outperforms previous watermarking methods in quality across various tasks.
  arXiv  Detail & Related papers  (2024-07-17T18:52:12Z)
• Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models [31.062753031312006]
  Large language models generate high-quality responses with potential misinformation. Watermarking is pivotal in this context, which involves embedding hidden markers in texts. We introduce a novel multi-objective optimization (MOO) approach for watermarking. Our method simultaneously achieves detectability and semantic integrity.
  arXiv  Detail & Related papers  (2024-02-28T05:43:22Z)
• GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick [50.35069175236422]
  Large language models (LLMs) excellently generate human-like text, but also raise concerns about misuse in fake news and academic dishonesty. Decoding-based watermark, particularly the GumbelMax-trick-based watermark(GM watermark), is a standout solution for safeguarding machine-generated texts. We propose a new type of GM watermark, the Logits-Addition watermark, and its three variants, specifically designed to enhance diversity.
  arXiv  Detail & Related papers  (2024-02-20T12:05:47Z)
• WatME: Towards Lossless Watermarking Through Lexical Redundancy [58.61972059246715]
  This study assesses the impact of watermarking on different capabilities of large language models (LLMs) from a cognitive science lens. We introduce Watermarking with Mutual Exclusion (WatME) to seamlessly integrate watermarks.
  arXiv  Detail & Related papers  (2023-11-16T11:58:31Z)
• Downstream Trade-offs of a Family of Text Watermarks [25.408313192999504]
  We evaluate the performance of LLMs watermarked using three different strategies over a diverse suite of tasks. We find that watermarks can cause significant drops in LLMs' effective utility across all tasks. Our findings highlight the trade-offs that users should be cognizant of when using watermarked models.
  arXiv  Detail & Related papers  (2023-11-16T11:44:58Z)
• Provable Robust Watermarking for AI-Generated Text [41.5510809722375]
  We propose a robust and high-quality watermark method, Unigram-Watermark. We prove that our watermark method enjoys guaranteed generation quality, correctness in watermark detection, and is robust against text editing and paraphrasing.
  arXiv  Detail & Related papers  (2023-06-30T07:24:32Z)
• On the Reliability of Watermarks for Large Language Models [95.87476978352659]
  We study the robustness of watermarked text after it is re-written by humans, paraphrased by a non-watermarked LLM, or mixed into a longer hand-written document. We find that watermarks remain detectable even after human and machine paraphrasing. We also consider a range of new detection schemes that are sensitive to short spans of watermarked text embedded inside a large document.
  arXiv  Detail & Related papers  (2023-06-07T17:58:48Z)
• A Watermark for Large Language Models [84.95327142027183]
  We propose a watermarking framework for proprietary language models. The watermark can be embedded with negligible impact on text quality. It can be detected using an efficient open-source algorithm without access to the language model API or parameters.
  arXiv  Detail & Related papers  (2023-01-24T18:52:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.