ESpeW: Robust Copyright Protection for LLM-based EaaS via Embedding-Specific Watermark

• URL: http://arxiv.org/abs/2410.17552v2
• Date: Thu, 24 Oct 2024 02:35:09 GMT
• Title: ESpeW: Robust Copyright Protection for LLM-based EaaS via Embedding-Specific Watermark
• Authors: Zongqi Wang, Baoyuan Wu, Jingyuan Deng, Yujiu Yang,
• Abstract summary: Embeds as a Service (Eding) is emerging as a crucial role in AI applications. Eding is vulnerable to model extraction attacks, highlighting the urgent need for copyright protection. We propose a novel embedding-specific watermarking (ESpeW) mechanism to offer robust copyright protection for Eding.
• Score: 50.08021440235581
• License:
• Abstract: Embeddings as a Service (EaaS) is emerging as a crucial role in AI applications. Unfortunately, EaaS is vulnerable to model extraction attacks, highlighting the urgent need for copyright protection. Although some preliminary works propose applying embedding watermarks to protect EaaS, recent research reveals that these watermarks can be easily removed. Hence, it is crucial to inject robust watermarks resistant to watermark removal attacks. Existing watermarking methods typically inject a target embedding into embeddings through linear interpolation when the text contains triggers. However, this mechanism results in each watermarked embedding having the same component, which makes the watermark easy to identify and eliminate. Motivated by this, in this paper, we propose a novel embedding-specific watermarking (ESpeW) mechanism to offer robust copyright protection for EaaS. Our approach involves injecting unique, yet readily identifiable watermarks into each embedding. Watermarks inserted by ESpeW are designed to maintain a significant distance from one another and to avoid sharing common components, thus making it significantly more challenging to remove the watermarks. Extensive experiments on four popular datasets demonstrate that ESpeW can even watermark successfully against a highly aggressive removal strategy without sacrificing the quality of embeddings. Code is available at https://github.com/liudan193/ESpeW.

Related papers

• Can Watermarked LLMs be Identified by Users via Crafted Prompts? [55.460327393792156]
  This work is the first to investigate the imperceptibility of watermarked Large Language Models (LLMs) We design an identification algorithm called Water-Probe that detects watermarks through well-designed prompts. Experiments show that almost all mainstream watermarking algorithms are easily identified with our well-designed prompts.
  arXiv  Detail & Related papers  (2024-10-04T06:01:27Z)
• Steganalysis on Digital Watermarking: Is Your Defense Truly Impervious? [21.06493827123594]
  steganalysis attacks can extract and remove the watermark with minimal perceptual distortion. We show how averaging a collection of watermarked images could reveal the underlying watermark pattern. We propose security guidelines calling for using content-adaptive watermarking strategies and performing security evaluation against steganalysis.
  arXiv  Detail & Related papers  (2024-06-13T12:01:28Z)
• Lost in Overlap: Exploring Watermark Collision in LLMs [6.398660996031915]
  We introduce watermark collision as a novel and general philosophy for watermark attacks. We provide a comprehensive demonstration that watermark collision poses a threat to all logit-based watermark algorithms.
  arXiv  Detail & Related papers  (2024-03-15T05:06:21Z)
• DeepEclipse: How to Break White-Box DNN-Watermarking Schemes [60.472676088146436]
  We present obfuscation techniques that significantly differ from the existing white-box watermarking removal schemes. DeepEclipse can evade watermark detection without prior knowledge of the underlying watermarking scheme. Our evaluation reveals that DeepEclipse excels in breaking multiple white-box watermarking schemes.
  arXiv  Detail & Related papers  (2024-03-06T10:24:47Z)
• WARDEN: Multi-Directional Backdoor Watermarks for Embedding-as-a-Service Copyright Protection [7.660430606056949]
  We propose a new protocol to make the removal of watermarks more challenging by incorporating multiple possible watermark directions. Our defense approach, WARDEN, notably increases the stealthiness of watermarks and has been empirically shown to be effective against CSE attack.
  arXiv  Detail & Related papers  (2024-03-03T10:39:27Z)
• Turning Your Strength into Watermark: Watermarking Large Language Model via Knowledge Injection [66.26348985345776]
  We propose a novel watermarking method for large language models (LLMs) based on knowledge injection. In the watermark embedding stage, we first embed the watermarks into the selected knowledge to obtain the watermarked knowledge. In the watermark extraction stage, questions related to the watermarked knowledge are designed, for querying the suspect LLM. Experiments show that the watermark extraction success rate is close to 100% and demonstrate the effectiveness, fidelity, stealthiness, and robustness of our proposed method.
  arXiv  Detail & Related papers  (2023-11-16T03:22:53Z)
• Supervised GAN Watermarking for Intellectual Property Protection [33.827150843939094]
  We propose a watermarking method for Generative Adversarial Networks (GANs) The aim is to watermark the GAN model so that any image generated by the GAN contains an invisible watermark (signature) Results show that our method can effectively embed an invisible watermark inside the generated images.
  arXiv  Detail & Related papers  (2022-09-07T20:52:05Z)
• Certified Neural Network Watermarks with Randomized Smoothing [64.86178395240469]
  We propose a certifiable watermarking method for deep learning models. We show that our watermark is guaranteed to be unremovable unless the model parameters are changed by more than a certain l2 threshold. Our watermark is also empirically more robust compared to previous watermarking methods.
  arXiv  Detail & Related papers  (2022-07-16T16:06:59Z)
• Fine-tuning Is Not Enough: A Simple yet Effective Watermark Removal Attack for DNN Models [72.9364216776529]
  We propose a novel watermark removal attack from a different perspective. We design a simple yet powerful transformation algorithm by combining imperceptible pattern embedding and spatial-level transformations. Our attack can bypass state-of-the-art watermarking solutions with very high success rates.
  arXiv  Detail & Related papers  (2020-09-18T09:14:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.