Watermark Stealing in Large Language Models

• URL: http://arxiv.org/abs/2402.19361v2
• Date: Mon, 24 Jun 2024 14:48:29 GMT
• Title: Watermark Stealing in Large Language Models
• Authors: Nikola Jovanović, Robin Staab, Martin Vechev,
• Abstract summary: We show that querying the API of the watermarked LLM to approximately reverse-engineer a watermark enables practical spoofing attacks. We are the first to propose an automated WS algorithm and use it in the first comprehensive study of spoofing and scrubbing in realistic settings.
• Score: 2.1165011830664673
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM watermarking has attracted attention as a promising way to detect AI-generated content, with some works suggesting that current schemes may already be fit for deployment. In this work we dispute this claim, identifying watermark stealing (WS) as a fundamental vulnerability of these schemes. We show that querying the API of the watermarked LLM to approximately reverse-engineer a watermark enables practical spoofing attacks, as hypothesized in prior work, but also greatly boosts scrubbing attacks, which was previously unnoticed. We are the first to propose an automated WS algorithm and use it in the first comprehensive study of spoofing and scrubbing in realistic settings. We show that for under $50 an attacker can both spoof and scrub state-of-the-art schemes previously considered safe, with average success rate of over 80%. Our findings challenge common beliefs about LLM watermarking, stressing the need for more robust schemes. We make all our code and additional examples available at https://watermark-stealing.org.

Related papers

• Your Fixed Watermark is Fragile: Towards Semantic-Aware Watermark for EaaS Copyright Protection [5.2431999629987]
  Embedding-as-a-Service (E) has emerged as a successful business pattern but faces significant challenges related to copyright infringement. Various studies have proposed backdoor-based watermarking schemes to protect the copyright of E services. In this paper, we reveal that previous watermarking schemes possess semantic-independent characteristics.
  arXiv  Detail & Related papers  (2024-11-14T11:06:34Z)
• Can Watermarked LLMs be Identified by Users via Crafted Prompts? [55.460327393792156]
  This work is the first to investigate the imperceptibility of watermarked Large Language Models (LLMs) We design an identification algorithm called Water-Probe that detects watermarks through well-designed prompts. Experiments show that almost all mainstream watermarking algorithms are easily identified with our well-designed prompts.
  arXiv  Detail & Related papers  (2024-10-04T06:01:27Z)
• Certifiably Robust Image Watermark [57.546016845801134]
  Generative AI raises many societal concerns such as boosting disinformation and propaganda campaigns. Watermarking AI-generated content is a key technology to address these concerns. We propose the first image watermarks with certified robustness guarantees against removal and forgery attacks.
  arXiv  Detail & Related papers  (2024-07-04T17:56:04Z)
• Large Language Model Watermark Stealing With Mixed Integer Programming [51.336009662771396]
  Large Language Model (LLM) watermark shows promise in addressing copyright, monitoring AI-generated text, and preventing its misuse. Recent research indicates that watermarking methods using numerous keys are susceptible to removal attacks. We propose a novel green list stealing attack against the state-of-the-art LLM watermark scheme.
  arXiv  Detail & Related papers  (2024-05-30T04:11:17Z)
• Black-Box Detection of Language Model Watermarks [1.9374282535132377]
  We develop rigorous statistical tests to detect the presence of all three most popular watermarking scheme families using only a limited number of black-box queries. Our findings indicate that current watermarking schemes are more detectable than previously believed, and that obscuring the fact that a watermark was deployed may not be a viable way for providers to protect against adversaries.
  arXiv  Detail & Related papers  (2024-05-28T08:41:30Z)
• Lost in Overlap: Exploring Watermark Collision in LLMs [6.398660996031915]
  We introduce watermark collision as a novel and general philosophy for watermark attacks. We provide a comprehensive demonstration that watermark collision poses a threat to all logit-based watermark algorithms.
  arXiv  Detail & Related papers  (2024-03-15T05:06:21Z)
• Turning Your Strength into Watermark: Watermarking Large Language Model via Knowledge Injection [66.26348985345776]
  We propose a novel watermarking method for large language models (LLMs) based on knowledge injection. In the watermark embedding stage, we first embed the watermarks into the selected knowledge to obtain the watermarked knowledge. In the watermark extraction stage, questions related to the watermarked knowledge are designed, for querying the suspect LLM. Experiments show that the watermark extraction success rate is close to 100% and demonstrate the effectiveness, fidelity, stealthiness, and robustness of our proposed method.
  arXiv  Detail & Related papers  (2023-11-16T03:22:53Z)
• A Semantic Invariant Robust Watermark for Large Language Models [27.522264953691746]
  Prior watermark algorithms face a trade-off between attack robustness and security robustness. This is because the watermark logits for a token are determined by a certain number of preceding tokens. We propose a semantic invariant watermarking method for LLMs that provides both attack robustness and security robustness.
  arXiv  Detail & Related papers  (2023-10-10T06:49:43Z)
• An Unforgeable Publicly Verifiable Watermark for Large Language Models [84.2805275589553]
  Current watermark detection algorithms require the secret key used in the watermark generation process, making them susceptible to security breaches and counterfeiting during public detection. We propose an unforgeable publicly verifiable watermark algorithm named UPV that uses two different neural networks for watermark generation and detection, instead of using the same key at both stages.
  arXiv  Detail & Related papers  (2023-07-30T13:43:27Z)
• Fine-tuning Is Not Enough: A Simple yet Effective Watermark Removal Attack for DNN Models [72.9364216776529]
  We propose a novel watermark removal attack from a different perspective. We design a simple yet powerful transformation algorithm by combining imperceptible pattern embedding and spatial-level transformations. Our attack can bypass state-of-the-art watermarking solutions with very high success rates.
  arXiv  Detail & Related papers  (2020-09-18T09:14:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.