Toward Breaking Watermarks in Distortion-free Large Language Models

• URL: http://arxiv.org/abs/2502.18608v1
• Date: Tue, 25 Feb 2025 19:52:55 GMT
• Title: Toward Breaking Watermarks in Distortion-free Large Language Models
• Authors: Shayleen Reynolds, Saheed Obitayo, Niccolò Dalmasso, Dung Daniel T. Ngo, Vamsi K. Potluru, Manuela Veloso,
• Abstract summary: We show that it is possible to "compromise" the LLM and carry out a "spoofing" attack.<n>Specifically, we propose a mixed integer linear programming framework that accurately estimates the secret key used for watermarking.
• Score: 11.922206306917435
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, LLM watermarking has emerged as an attractive safeguard against AI-generated content, with promising applications in many real-world domains. However, there are growing concerns that the current LLM watermarking schemes are vulnerable to expert adversaries wishing to reverse-engineer the watermarking mechanisms. Prior work in "breaking" or "stealing" LLM watermarks mainly focuses on the distribution-modifying algorithm of Kirchenbauer et al. (2023), which perturbs the logit vector before sampling. In this work, we focus on reverse-engineering the other prominent LLM watermarking scheme, distortion-free watermarking (Kuditipudi et al. 2024), which preserves the underlying token distribution by using a hidden watermarking key sequence. We demonstrate that, even under a more sophisticated watermarking scheme, it is possible to "compromise" the LLM and carry out a "spoofing" attack. Specifically, we propose a mixed integer linear programming framework that accurately estimates the secret key used for watermarking using only a few samples of the watermarked dataset. Our initial findings challenge the current theoretical claims on the robustness and usability of existing LLM watermarking techniques.

Related papers

• ESpeW: Robust Copyright Protection for LLM-based EaaS via Embedding-Specific Watermark [50.08021440235581]
  Embeds as a Service (Eding) is emerging as a crucial role in AI applications. Eding is vulnerable to model extraction attacks, highlighting the urgent need for copyright protection. We propose a novel embedding-specific watermarking (ESpeW) mechanism to offer robust copyright protection for Eding.
  arXiv  Detail & Related papers  (2024-10-23T04:34:49Z)
• Can Watermarked LLMs be Identified by Users via Crafted Prompts? [55.460327393792156]
  This work is the first to investigate the imperceptibility of watermarked Large Language Models (LLMs)<n>We design an identification algorithm called Water-Probe that detects watermarks through well-designed prompts.<n> Experiments show that almost all mainstream watermarking algorithms are easily identified with our well-designed prompts.
  arXiv  Detail & Related papers  (2024-10-04T06:01:27Z)
• Large Language Model Watermark Stealing With Mixed Integer Programming [51.336009662771396]
  Large Language Model (LLM) watermark shows promise in addressing copyright, monitoring AI-generated text, and preventing its misuse. Recent research indicates that watermarking methods using numerous keys are susceptible to removal attacks. We propose a novel green list stealing attack against the state-of-the-art LLM watermark scheme.
  arXiv  Detail & Related papers  (2024-05-30T04:11:17Z)
• Watermark Stealing in Large Language Models [2.1165011830664673]
  We show that querying the API of the watermarked LLM to approximately reverse-engineer a watermark enables practical spoofing attacks. We are the first to propose an automated WS algorithm and use it in the first comprehensive study of spoofing and scrubbing in realistic settings.
  arXiv  Detail & Related papers  (2024-02-29T17:12:39Z)
• No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices [20.20770405297239]
  We show that common design choices in LLM watermarking schemes make the resulting systems surprisingly susceptible to attack. We propose guidelines and defenses for LLM watermarking in practice.
  arXiv  Detail & Related papers  (2024-02-25T20:24:07Z)
• Turning Your Strength into Watermark: Watermarking Large Language Model via Knowledge Injection [66.26348985345776]
  We propose a novel watermarking method for large language models (LLMs) based on knowledge injection. In the watermark embedding stage, we first embed the watermarks into the selected knowledge to obtain the watermarked knowledge. In the watermark extraction stage, questions related to the watermarked knowledge are designed, for querying the suspect LLM. Experiments show that the watermark extraction success rate is close to 100% and demonstrate the effectiveness, fidelity, stealthiness, and robustness of our proposed method.
  arXiv  Detail & Related papers  (2023-11-16T03:22:53Z)
• Unbiased Watermark for Large Language Models [67.43415395591221]
  This study examines how significantly watermarks impact the quality of model-generated outputs. It is possible to integrate watermarks without affecting the output probability distribution. The presence of watermarks does not compromise the performance of the model in downstream tasks.
  arXiv  Detail & Related papers  (2023-09-22T12:46:38Z)
• Fine-tuning Is Not Enough: A Simple yet Effective Watermark Removal Attack for DNN Models [72.9364216776529]
  We propose a novel watermark removal attack from a different perspective. We design a simple yet powerful transformation algorithm by combining imperceptible pattern embedding and spatial-level transformations. Our attack can bypass state-of-the-art watermarking solutions with very high success rates.
  arXiv  Detail & Related papers  (2020-09-18T09:14:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.