Watermarking Visual Concepts for Diffusion Models

• URL: http://arxiv.org/abs/2411.11688v3
• Date: Sat, 23 Aug 2025 12:04:45 GMT
• Title: Watermarking Visual Concepts for Diffusion Models
• Authors: Liangqi Lei, Keke Gai, Jing Yu, Liehuang Zhu, Qi Wu,
• Abstract summary: Personalization techniques generate images with specific concepts.<n> malicious users can generate unauthorized content and disinformation relevant to a target concept.<n>Model watermarking is an effective solution to trace the malicious generated images and safeguard their copyright.
• Score: 43.35783380047233
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: The personalization techniques of diffusion models succeed in generating images with specific concepts. This ability also poses great threats to copyright protection and network security since malicious users can generate unauthorized content and disinformation relevant to a target concept. Model watermarking is an effective solution to trace the malicious generated images and safeguard their copyright. However, existing model watermarking techniques merely achieve image-level tracing without concept traceability. When tracing infringing or harmful concepts, current approaches execute image concept detection and model tracing sequentially, where performance is critically constrained by concept detection accuracy. In this paper, we propose a lightweight concept watermarking framework that efficiently binds target concepts to model watermarks, supporting simultaneous concept identification and model tracing via single-stage watermark verification. To further enhance the robustness of concept watermarking, we propose an adversarial perturbation injection method collaboratively embedded with watermarks during image generation, avoiding watermark removal by model purification attacks. Experimental results demonstrate that ConceptWM significantly outperforms state-of-the-art watermarking methods, improving detection accuracy by 6.3%-19.3% across diverse datasets including COCO and StableDiffusionDB. Additionally, ConceptWM possesses a critical capability absent in other watermarking methods: it sustains a 21.7% FID/CLIP degradation under adversarial fine-tuning of Stable Diffusion models on WikiArt and CelebA-HQ, demonstrating its capability to mitigate model misuse.

Related papers

• Vanishing Watermarks: Diffusion-Based Image Editing Undermines Robust Invisible Watermarking [3.583615559438432]
  Powerful diffusion-based image generation and editing techniques now pose a new threat to robust invisible watermarking schemes.<n>We show that diffusion models can effectively erase robust watermarks even when those watermarks were designed to withstand conventional distortions.<n>We introduce a guided diffusion-based attack that explicitly targets the embedded watermark signal during generation, significantly degrading watermark detectability.
  arXiv  Detail & Related papers  (2026-02-24T08:34:48Z)
• Diffusion-Based Image Editing: An Unforeseen Adversary to Robust Invisible Watermarks [4.138397555991069]
  Powerful diffusion-based image generation and editing models can inadvertently remove or distort embedded watermarks.<n>We present a theoretical and empirical analysis demonstrating that diffusion-based image editing can effectively break state-of-the-art robust watermarks.<n>We propose a diffusion-driven attack that uses generative image regeneration to erase watermarks from a given image.
  arXiv  Detail & Related papers  (2025-11-05T16:20:29Z)
• Diffusion-Based Image Editing for Breaking Robust Watermarks [4.273350357872755]
  Powerful diffusion-based image generation and editing techniques pose a new threat to robust watermarking schemes.<n>We show that a diffusion-driven image regeneration'' process can erase embedded watermarks while preserving image content.<n>We introduce a novel guided diffusion attack that explicitly targets the watermark signal during generation, significantly degrading watermark detectability.
  arXiv  Detail & Related papers  (2025-10-07T14:34:42Z)
• StableGuard: Towards Unified Copyright Protection and Tamper Localization in Latent Diffusion Models [55.05404953041403]
  We propose a novel framework that seamlessly integrates a binary watermark into the diffusion generation process.<n>We show that StableGuard consistently outperforms state-of-the-art methods in image fidelity, watermark verification, and tampering localization.
  arXiv  Detail & Related papers  (2025-09-22T16:35:19Z)
• Optimization-Free Universal Watermark Forgery with Regenerative Diffusion Models [50.73220224678009]
  Watermarking can be used to verify the origin of synthetic images generated by artificial intelligence models.<n>Recent studies demonstrate the capability to forge watermarks from a target image onto cover images via adversarial techniques.<n>In this paper, we uncover a greater risk of an optimization-free and universal watermark forgery.<n>Our approach significantly broadens the scope of attacks, presenting a greater challenge to the security of current watermarking techniques.
  arXiv  Detail & Related papers  (2025-06-06T12:08:02Z)
• Forging and Removing Latent-Noise Diffusion Watermarks Using a Single Image [24.513881574366046]
  Previous watermarking schemes embed a secret key in the initial noise. We propose a black-box adversarial attack without presuming access to the diffusion model weights. We show that we can also apply a similar approach for watermark removal by learning perturbations to exit this region.
  arXiv  Detail & Related papers  (2025-04-27T18:29:26Z)
• Secure and Efficient Watermarking for Latent Diffusion Models in Model Distribution Scenarios [23.64920988914223]
  A new security mechanism is designed to prevent watermark leakage and watermark escape.<n>A watermark distribution-based verification strategy is proposed to enhance the robustness against diverse attacks in the model distribution scenarios.
  arXiv  Detail & Related papers  (2025-02-18T23:55:33Z)
• SWA-LDM: Toward Stealthy Watermarks for Latent Diffusion Models [11.906245347904289]
  We introduce SWA-LDM, a novel approach that enhances watermarking by randomizing the embedding process.<n>Our proposed watermark presence attack reveals the inherent vulnerabilities of existing latent-based watermarking methods.<n>This work represents a pivotal step towards securing LDM-generated images against unauthorized use.
  arXiv  Detail & Related papers  (2025-02-14T16:55:45Z)
• Spread them Apart: Towards Robust Watermarking of Generated Content [4.332441337407564]
  We propose an approach to embed watermarks into the generated content to allow future detection of the generated content and identification of the user who generated it. We prove that watermarks embedded are guaranteed to be robust against additive perturbations of a bounded magnitude.
  arXiv  Detail & Related papers  (2025-02-11T09:23:38Z)
• RoboSignature: Robust Signature and Watermarking on Network Attacks [0.5461938536945723]
  We present a novel adversarial fine-tuning attack that disrupts the model's ability to embed the intended watermark. Our findings emphasize the importance of anticipating and defending against potential vulnerabilities in generative systems.
  arXiv  Detail & Related papers  (2024-12-22T04:36:27Z)
• Exploiting Watermark-Based Defense Mechanisms in Text-to-Image Diffusion Models for Unauthorized Data Usage [14.985938758090763]
  Text-to-image diffusion models, such as Stable Diffusion, have shown exceptional potential in generating high-quality images. Recent studies highlight concerns over the use of unauthorized data in training these models, which may lead to intellectual property infringement or privacy violations. In this paper, we examine the robustness of various watermark-based protection methods applied to text-to-image models.
  arXiv  Detail & Related papers  (2024-11-22T22:28:19Z)
• ESpeW: Robust Copyright Protection for LLM-based EaaS via Embedding-Specific Watermark [50.08021440235581]
  Embeds as a Service (Eding) is emerging as a crucial role in AI applications. Eding is vulnerable to model extraction attacks, highlighting the urgent need for copyright protection. We propose a novel embedding-specific watermarking (ESpeW) mechanism to offer robust copyright protection for Eding.
  arXiv  Detail & Related papers  (2024-10-23T04:34:49Z)
• JIGMARK: A Black-Box Approach for Enhancing Image Watermarks against Diffusion Model Edits [76.25962336540226]
  JIGMARK is a first-of-its-kind watermarking technique that enhances robustness through contrastive learning. Our evaluation reveals that JIGMARK significantly surpasses existing watermarking solutions in resilience to diffusion-model edits.
  arXiv  Detail & Related papers  (2024-06-06T03:31:41Z)
• AquaLoRA: Toward White-box Protection for Customized Stable Diffusion Models via Watermark LoRA [67.68750063537482]
  Diffusion models have achieved remarkable success in generating high-quality images. Recent works aim to let SD models output watermarked content for post-hoc forensics. We propose textttmethod as the first implementation under this scenario.
  arXiv  Detail & Related papers  (2024-05-18T01:25:47Z)
• DeepEclipse: How to Break White-Box DNN-Watermarking Schemes [60.472676088146436]
  We present obfuscation techniques that significantly differ from the existing white-box watermarking removal schemes. DeepEclipse can evade watermark detection without prior knowledge of the underlying watermarking scheme. Our evaluation reveals that DeepEclipse excels in breaking multiple white-box watermarking schemes.
  arXiv  Detail & Related papers  (2024-03-06T10:24:47Z)
• FT-Shield: A Watermark Against Unauthorized Fine-tuning in Text-to-Image Diffusion Models [64.89896692649589]
  We propose FT-Shield, a watermarking solution tailored for the fine-tuning of text-to-image diffusion models. FT-Shield addresses copyright protection challenges by designing new watermark generation and detection strategies.
  arXiv  Detail & Related papers  (2023-10-03T19:50:08Z)
• Catch You Everything Everywhere: Guarding Textual Inversion via Concept Watermarking [67.60174799881597]
  We propose the novel concept watermarking, where watermark information is embedded into the target concept and then extracted from generated images based on the watermarked concept. In practice, the concept owner can upload his concept with different watermarks (ie, serial numbers) to the platform, and the platform allocates different users with different serial numbers for subsequent tracing and forensics.
  arXiv  Detail & Related papers  (2023-09-12T03:33:13Z)
• Invisible Image Watermarks Are Provably Removable Using Generative AI [47.25747266531665]
  Invisible watermarks safeguard images' copyrights by embedding hidden messages only detectable by owners. We propose a family of regeneration attacks to remove these invisible watermarks. The proposed attack method first adds random noise to an image to destroy the watermark and then reconstructs the image.
  arXiv  Detail & Related papers  (2023-06-02T23:29:28Z)
• Certified Neural Network Watermarks with Randomized Smoothing [64.86178395240469]
  We propose a certifiable watermarking method for deep learning models. We show that our watermark is guaranteed to be unremovable unless the model parameters are changed by more than a certain l2 threshold. Our watermark is also empirically more robust compared to previous watermarking methods.
  arXiv  Detail & Related papers  (2022-07-16T16:06:59Z)
• Exploring Structure Consistency for Deep Model Watermarking [122.38456787761497]
  The intellectual property (IP) of Deep neural networks (DNNs) can be easily stolen'' by surrogate model attack. We propose a new watermarking methodology, namely structure consistency'', based on which a new deep structure-aligned model watermarking algorithm is designed.
  arXiv  Detail & Related papers  (2021-08-05T04:27:15Z)
• Fine-tuning Is Not Enough: A Simple yet Effective Watermark Removal Attack for DNN Models [72.9364216776529]
  We propose a novel watermark removal attack from a different perspective. We design a simple yet powerful transformation algorithm by combining imperceptible pattern embedding and spatial-level transformations. Our attack can bypass state-of-the-art watermarking solutions with very high success rates.
  arXiv  Detail & Related papers  (2020-09-18T09:14:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.