AGATE: Stealthy Black-box Watermarking for Multimodal Model Copyright Protection

• URL: http://arxiv.org/abs/2504.21044v1
• Date: Mon, 28 Apr 2025 14:52:01 GMT
• Title: AGATE: Stealthy Black-box Watermarking for Multimodal Model Copyright Protection
• Authors: Jianbo Gao, Keke Gai, Jing Yu, Liehuang Zhu, Qi Wu,
• Abstract summary: Methods select Out-of-Distribution (OoD) data as backdoor watermarks and retrain the original model for copyright protection.<n>Existing methods are susceptible to malicious detection and forgery by adversaries, resulting in watermark evasion.<n>We propose Model-underlineagnostic Black-box Backdoor Wunderlineatermarking Framework (AGATE) to address stealthiness and robustness challenges in multimodal model copyright protection.
• Score: 26.066755429896926
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advancement in large-scale Artificial Intelligence (AI) models offering multimodal services have become foundational in AI systems, making them prime targets for model theft. Existing methods select Out-of-Distribution (OoD) data as backdoor watermarks and retrain the original model for copyright protection. However, existing methods are susceptible to malicious detection and forgery by adversaries, resulting in watermark evasion. In this work, we propose Model-\underline{ag}nostic Black-box Backdoor W\underline{ate}rmarking Framework (AGATE) to address stealthiness and robustness challenges in multimodal model copyright protection. Specifically, we propose an adversarial trigger generation method to generate stealthy adversarial triggers from ordinary dataset, providing visual fidelity while inducing semantic shifts. To alleviate the issue of anomaly detection among model outputs, we propose a post-transform module to correct the model output by narrowing the distance between adversarial trigger image embedding and text embedding. Subsequently, a two-phase watermark verification is proposed to judge whether the current model infringes by comparing the two results with and without the transform module. Consequently, we consistently outperform state-of-the-art methods across five datasets in the downstream tasks of multimodal image-text retrieval and image classification. Additionally, we validated the robustness of AGATE under two adversarial attack scenarios.

Related papers

• SleeperMark: Towards Robust Watermark against Fine-Tuning Text-to-image Diffusion Models [77.80595722480074]
  SleeperMark is a framework designed to embed resilient watermarks into T2I diffusion models.<n>It guides the model to disentangle the watermark information from the semantic concepts it learns.<n>Our experiments demonstrate the effectiveness of SleeperMark across various types of diffusion models.
  arXiv  Detail & Related papers  (2024-12-06T08:44:18Z)
• Explanation as a Watermark: Towards Harmless and Multi-bit Model Ownership Verification via Watermarking Feature Attribution [22.933101948176606]
  backdoor-based model watermarks are the primary and cutting-edge methods to implant such properties in released models. We design a new watermarking paradigm, $i.e.$, Explanation as a Watermark (EaaW), that implants verification behaviors into the explanation of feature attribution.
  arXiv  Detail & Related papers  (2024-05-08T05:49:46Z)
• ModelShield: Adaptive and Robust Watermark against Model Extraction Attack [58.46326901858431]
  Large language models (LLMs) demonstrate general intelligence across a variety of machine learning tasks.<n> adversaries can still utilize model extraction attacks to steal the model intelligence encoded in model generation.<n> Watermarking technology offers a promising solution for defending against such attacks by embedding unique identifiers into the model-generated content.
  arXiv  Detail & Related papers  (2024-05-03T06:41:48Z)
• Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models [71.13610023354967]
  Copyright protection and inappropriate content generation pose challenges for the practical implementation of diffusion models. We propose a diffusion model watermarking technique that is both performance-lossless and training-free.
  arXiv  Detail & Related papers  (2024-04-07T13:30:10Z)
• Towards Robust Model Watermark via Reducing Parametric Vulnerability [57.66709830576457]
  backdoor-based ownership verification becomes popular recently, in which the model owner can watermark the model. We propose a mini-max formulation to find these watermark-removed models and recover their watermark behavior. Our method improves the robustness of the model watermarking against parametric changes and numerous watermark-removal attacks.
  arXiv  Detail & Related papers  (2023-09-09T12:46:08Z)
• Seeds Don't Lie: An Adaptive Watermarking Framework for Computer Vision Models [44.80560808267494]
  We present an adaptive framework to watermark a protected model, leveraging the unique behavior present in the model. This watermark is used to detect extracted models, which have the same unique behavior, indicating an unauthorized usage of the protected model's IP. We show that the framework is robust to (1) unseen model extraction attacks, and (2) extracted models which undergo a method (e.g., weight pruning)
  arXiv  Detail & Related papers  (2022-11-24T14:48:40Z)
• Are You Stealing My Model? Sample Correlation for Fingerprinting Deep Neural Networks [86.55317144826179]
  Previous methods always leverage the transferable adversarial examples as the model fingerprint. We propose a novel yet simple model stealing detection method based on SAmple Correlation (SAC) SAC successfully defends against various model stealing attacks, even including adversarial training or transfer learning.
  arXiv  Detail & Related papers  (2022-10-21T02:07:50Z)
• DeepHider: A Multi-module and Invisibility Watermarking Scheme for Language Model [0.0]
  This paper proposes a new threat of replacing the model classification module and performing global fine-tuning of the model. We use the properties of blockchain such as tamper-proof and traceability to prevent the ownership statement of thieves. Experiments show that the proposed scheme successfully verifies ownership with 100% watermark verification accuracy.
  arXiv  Detail & Related papers  (2022-08-09T11:53:24Z)
• DynaMarks: Defending Against Deep Learning Model Extraction Using Dynamic Watermarking [3.282282297279473]
  The functionality of a deep learning (DL) model can be stolen via model extraction. We propose a novel watermarking technique called DynaMarks to protect the intellectual property (IP) of DL models.
  arXiv  Detail & Related papers  (2022-07-27T06:49:39Z)
• Exploring Structure Consistency for Deep Model Watermarking [122.38456787761497]
  The intellectual property (IP) of Deep neural networks (DNNs) can be easily stolen'' by surrogate model attack. We propose a new watermarking methodology, namely structure consistency'', based on which a new deep structure-aligned model watermarking algorithm is designed.
  arXiv  Detail & Related papers  (2021-08-05T04:27:15Z)
• Model Watermarking for Image Processing Networks [120.918532981871]
  How to protect the intellectual property of deep models is a very important but seriously under-researched problem. We propose the first model watermarking framework for protecting image processing models.
  arXiv  Detail & Related papers  (2020-02-25T18:36:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.