Optimizing watermarks for large language models

• URL: http://arxiv.org/abs/2312.17295v1
• Date: Thu, 28 Dec 2023 16:10:51 GMT
• Title: Optimizing watermarks for large language models
• Authors: Bram Wouters
• Abstract summary: This paper introduces a systematic approach to the trade-off between watermark identifiability and their impact on the quality of the generated text. For a large class of robust, efficient watermarks, the associated optimal solutions are identified and shown to outperform the currently default watermark.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the rise of large language models (LLMs) and concerns about potential misuse, watermarks for generative LLMs have recently attracted much attention. An important aspect of such watermarks is the trade-off between their identifiability and their impact on the quality of the generated text. This paper introduces a systematic approach to this trade-off in terms of a multi-objective optimization problem. For a large class of robust, efficient watermarks, the associated Pareto optimal solutions are identified and shown to outperform the currently default watermark.

Related papers

• Improved Unbiased Watermark for Large Language Models [59.00698153097887]
  We introduce MCmark, a family of unbiased, Multi-Channel-based watermarks. MCmark preserves the original distribution of the language model. It offers significant improvements in detectability and robustness over existing unbiased watermarks.
  arXiv  Detail & Related papers  (2025-02-16T21:02:36Z)
• De-mark: Watermark Removal in Large Language Models [59.00698153097887]
  We present De-mark, an advanced framework designed to remove n-gram-based watermarks effectively. Our method utilizes a novel querying strategy, termed random selection probing, which aids in assessing the strength of the watermark.
  arXiv  Detail & Related papers  (2024-10-17T17:42:10Z)
• Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approach [35.319577498993354]
  We present a novel theoretical framework for watermarking Large Language Models (LLMs) Our approach focuses on maximizing detection performance while maintaining control over the worst-case Type-I error and text distortion. We propose an efficient, model-agnostic, distribution-adaptive watermarking algorithm, utilizing a surrogate model alongside the Gumbel-max trick.
  arXiv  Detail & Related papers  (2024-10-03T18:28:10Z)
• Towards Effective User Attribution for Latent Diffusion Models via Watermark-Informed Blending [54.26862913139299]
  We introduce a novel framework Towards Effective user Attribution for latent diffusion models via Watermark-Informed Blending (TEAWIB) TEAWIB incorporates a unique ready-to-use configuration approach that allows seamless integration of user-specific watermarks into generative models. Experiments validate the effectiveness of TEAWIB, showcasing the state-of-the-art performance in perceptual quality and attribution accuracy.
  arXiv  Detail & Related papers  (2024-09-17T07:52:09Z)
• WaterJudge: Quality-Detection Trade-off when Watermarking Large Language Models [36.92452515593206]
  This paper proposes a simple analysis framework where comparative assessment, a flexible NLG evaluation framework, is used to assess the quality degradation caused by a particular watermark setting. We demonstrate that our framework provides easy visualization of the quality-detection trade-off of watermark settings. This approach is applied to two different summarization systems and a translation system, enabling cross-model analysis for a task, and cross-task analysis.
  arXiv  Detail & Related papers  (2024-03-28T16:28:38Z)
• Cross-Attention Watermarking of Large Language Models [8.704964543257246]
  New approach to linguistic watermarking of language models is presented. Information is imperceptibly inserted into the output text while preserving its readability and original meaning. Cross-attention mechanism is used to embed watermarks in the text during inference.
  arXiv  Detail & Related papers  (2024-01-12T09:39:50Z)
• WatME: Towards Lossless Watermarking Through Lexical Redundancy [58.61972059246715]
  This study assesses the impact of watermarking on different capabilities of large language models (LLMs) from a cognitive science lens. We introduce Watermarking with Mutual Exclusion (WatME) to seamlessly integrate watermarks.
  arXiv  Detail & Related papers  (2023-11-16T11:58:31Z)
• Improving the Generation Quality of Watermarked Large Language Models via Word Importance Scoring [81.62249424226084]
  Token-level watermarking inserts watermarks in the generated texts by altering the token probability distributions. This watermarking algorithm alters the logits during generation, which can lead to a downgraded text quality. We propose to improve the quality of texts generated by a watermarked language model by Watermarking with Importance Scoring (WIS)
  arXiv  Detail & Related papers  (2023-11-16T08:36:00Z)
• Unbiased Watermark for Large Language Models [67.43415395591221]
  This study examines how significantly watermarks impact the quality of model-generated outputs. It is possible to integrate watermarks without affecting the output probability distribution. The presence of watermarks does not compromise the performance of the model in downstream tasks.
  arXiv  Detail & Related papers  (2023-09-22T12:46:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.