Vaporizer: Breaking Watermarking Schemes for Large Language Model Outputs

• URL: http://arxiv.org/abs/2605.07481v1
• Date: Fri, 08 May 2026 09:24:29 GMT
• Title: Vaporizer: Breaking Watermarking Schemes for Large Language Model Outputs
• Abstract summary: We investigate the recent state-of-the-art schemes for watermarking large language models (LLMs) outputs.<n>We analyse the effectiveness of these watermarking techniques against an extensive collection of modified text attacks.
• Score: 2.5756681494057045
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we investigate the recent state-of-the-art schemes for watermarking large language models (LLMs) outputs. These techniques are claimed to be robust, scalable and production-grade, aimed at promoting responsible usage of LLMs. We analyse the effectiveness of these watermarking techniques against an extensive collection of modified text attacks, which perform targeted semantic changes without altering the general meaning of the text content. Our approach encompasses multiple attack strategies, which include lexical alterations, machine translation, and even neural paraphrasing. The attack efficacy is measured with two target criteria - successful removal of the watermark and preservation of semantic content. We evaluate semantic preservation through BERT scores, text complexity measures, grammatical errors, and Flesch Reading Ease indices. The experimental results reveal varying levels of effectiveness among different watermarking models, with the same underlying result that it is possible to remove the watermark with reasonable effort. This study sheds light on the strengths and weaknesses of existing LLM watermarking systems, suggesting how they should be constructed to improve security of available schemes.

Related papers

• Robust Text Watermarking for Large Language Models via Dual Semantic Embeddings [2.992414059774663]
  This work presents a semantic watermarking scheme for large language models (LLMs)<n> Dual-Embedding Watermarking (DEW) uses contextual and token-level embeddings to enhance robustness against paraphrasing and translation.
  arXiv  Detail & Related papers  (2026-06-30T12:51:30Z)
• Watermarks for Embeddings-as-a-Service Large Language Models [4.230349220081445]
  This thesis focuses on defending against imitation attacks by investigating E watermarks.<n>Existing E watermarks can be removed through paraphrasing text when attackers clone the model during imitation attacks.<n>We propose a novel watermarking technique, which employs linear transformation embeddings.
  arXiv  Detail & Related papers  (2025-11-28T00:52:40Z)
• Character-Level Perturbations Disrupt LLM Watermarks [64.60090923837701]
  We formalize the system model for Large Language Model (LLM) watermarking.<n>We characterize two realistic threat models constrained on limited access to the watermark detector.<n>We demonstrate character-level perturbations are significantly more effective for watermark removal under the most restrictive threat model.<n> Experiments confirm the superiority of character-level perturbations and the effectiveness of the Genetic Algorithm (GA) in removing watermarks under realistic constraints.
  arXiv  Detail & Related papers  (2025-09-11T02:50:07Z)
• In-Context Watermarks for Large Language Models [71.29952527565749]
  In-Context Watermarking (ICW) embeds watermarks into generated text solely through prompt engineering.<n>We investigate four ICW strategies at different levels of granularity, each paired with a tailored detection method.<n>Our experiments validate the feasibility of ICW as a model-agnostic, practical watermarking approach.
  arXiv  Detail & Related papers  (2025-05-22T17:24:51Z)
• BiMarker: Enhancing Text Watermark Detection for Large Language Models with Bipolar Watermarks [13.741307434082033]
  Existing watermarking techniques struggle with low watermark strength and stringent false-positive requirements.<n>tool splits generated text into positive and negative poles, enhancing detection without requiring additional computational resources.
  arXiv  Detail & Related papers  (2025-01-21T14:32:50Z)
• Large Language Model Watermark Stealing With Mixed Integer Programming [51.336009662771396]
  Large Language Model (LLM) watermark shows promise in addressing copyright, monitoring AI-generated text, and preventing its misuse. Recent research indicates that watermarking methods using numerous keys are susceptible to removal attacks. We propose a novel green list stealing attack against the state-of-the-art LLM watermark scheme.
  arXiv  Detail & Related papers  (2024-05-30T04:11:17Z)
• Adaptive Text Watermark for Large Language Models [8.100123266517299]
  It is challenging to generate high-quality watermarked text while maintaining strong security, robustness, and the ability to detect watermarks without prior knowledge of the prompt or model. This paper proposes an adaptive watermarking strategy to address this problem.
  arXiv  Detail & Related papers  (2024-01-25T03:57:12Z)
• WatME: Towards Lossless Watermarking Through Lexical Redundancy [58.61972059246715]
  This study assesses the impact of watermarking on different capabilities of large language models (LLMs) from a cognitive science lens. We introduce Watermarking with Mutual Exclusion (WatME) to seamlessly integrate watermarks.
  arXiv  Detail & Related papers  (2023-11-16T11:58:31Z)
• Improving the Generation Quality of Watermarked Large Language Models via Word Importance Scoring [81.62249424226084]
  Token-level watermarking inserts watermarks in the generated texts by altering the token probability distributions. This watermarking algorithm alters the logits during generation, which can lead to a downgraded text quality. We propose to improve the quality of texts generated by a watermarked language model by Watermarking with Importance Scoring (WIS)
  arXiv  Detail & Related papers  (2023-11-16T08:36:00Z)
• A Robust Semantics-based Watermark for Large Language Model against Paraphrasing [50.84892876636013]
  Large language models (LLMs) have show great ability in various natural language tasks. There are concerns that LLMs are possible to be used improperly or even illegally. We propose a semantics-based watermark framework SemaMark.
  arXiv  Detail & Related papers  (2023-11-15T06:19:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.