Robust Watermarks Meet Backdoored Models: Evading Diffusion Semantic Watermarks via Stealthy Backdoor

• URL: http://arxiv.org/abs/2608.00543v1
• Date: Sat, 01 Aug 2026 08:58:15 GMT
• Title: Robust Watermarks Meet Backdoored Models: Evading Diffusion Semantic Watermarks via Stealthy Backdoor
• Abstract summary: GhostVAE is a stealthy backdoor into the encoder of Variational Autoencoder (VAE)<n>We show that GhostVAE preserves watermark detection performance on benign images.<n>We also demonstrate that GhostVAE remains stealthy across the input space, parameter space, and latent space.
• Score: 24.332611409003878
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although semantic watermarking is considered a promising safeguard for images generated by Latent Diffusion Models (LDMs), the reliance of the watermark detection pipeline on neural networks introduces a critical yet underexplored backdoor attack surface. To systematically study this vulnerability, we propose GhostVAE to plant a stealthy backdoor into the encoder of Variational Autoencoder (VAE), enabling reliable evasion of watermark detection. GhostVAE operates in two stages: it first constructs a universal trigger via power spectrum regularization to improve the trigger robustness, and then trains a backdoored VAE encoder with a parameter-aligned objective. Through extensive evaluations across three state-of-the-art semantic watermarking schemes and three widely adopted LDMs, we show that GhostVAE preserves watermark detection performance on benign images (achieving an average true positive rate of 94.4%), while simultaneously enabling highly effective evasion under trigger activation (achieving an average attack success rate of 94.6%). Moreover, we comprehensively analyze seventeen representative defenses and demonstrate that GhostVAE remains stealthy across the input space, parameter space, and latent space. Our work fundamentally undermines the trustworthiness of semantic watermarking systems and highlights that secure deployment of semantic watermarks requires end-to-end security considerations, particularly for neural network components.

Related papers

• PGID: Progressive Guided Inversion and Denoising for Robust Watermark Detection [2.9276293999058898]
  dependence of semantic watermarking on diffusion inversion for watermark detection creates a critical vulnerability.<n>We propose Progressive Guided Inversion and Denoising (PGID), the first plug-and-play, training-free noise extraction framework designed to defend against both attack strategies.
  arXiv  Detail & Related papers  (2026-05-10T04:32:57Z)
• SemBind: Binding Diffusion Watermarks to Semantics Against Black-Box Forgery Attacks [74.76909939060833]
  Black-box forgery attacks pose outsized risk to provenance and trust.<n>We propose SemBind, a framework for latent-based watermarks that resists black-box forgery.<n>We show that SemBind-enabled anti-forgery variants markedly reduce false acceptance under black-box forgery.
  arXiv  Detail & Related papers  (2026-01-28T07:02:40Z)
• DeMark: A Query-Free Black-Box Attack on Deepfake Watermarking Defenses [25.492274324587058]
  DeMark is a query-free black-box attack framework that targets defensive image watermarking schemes for deepfakes.<n>We exploit latent-space vulnerabilities in encoder-decoder watermarking models through a compressive sensing based sparsification process.<n>DeMark reduces watermark detection accuracy from 100% to 32.9% on average while maintaining natural visual quality.
  arXiv  Detail & Related papers  (2026-01-23T06:04:43Z)
• A Visual Semantic Adaptive Watermark grounded by Prefix-Tuning for Large Vision-Language Model [48.79816664229285]
  VIsual Semantic Adaptive Watermark (VISA-Mark) is a novel framework that embeds detectable signals while strictly preserving visual fidelity.<n>Our approach employs a lightweight, efficiently trained prefix-tuner to extract dynamic Visual-Evidence Weights.<n> Empirical results confirm that VISA-Mark outperforms conventional methods with a 7.8% improvement in visual consistency.
  arXiv  Detail & Related papers  (2026-01-12T07:55:13Z)
• StableGuard: Towards Unified Copyright Protection and Tamper Localization in Latent Diffusion Models [55.05404953041403]
  We propose a novel framework that seamlessly integrates a binary watermark into the diffusion generation process.<n>We show that StableGuard consistently outperforms state-of-the-art methods in image fidelity, watermark verification, and tampering localization.
  arXiv  Detail & Related papers  (2025-09-22T16:35:19Z)
• Semantic Watermarking Reinvented: Enhancing Robustness and Generation Quality with Fourier Integrity [31.666430190864947]
  We propose a novel embedding method called Hermitian Symmetric Fourier Watermarking (SFW)<n>SFW maintains frequency integrity by enforcing Hermitian symmetry.<n>We introduce a center-aware embedding strategy that reduces the vulnerability of semantic watermarking due to cropping attacks.
  arXiv  Detail & Related papers  (2025-09-09T12:15:16Z)
• Robust Watermarks Leak: Channel-Aware Feature Extraction Enables Adversarial Watermark Manipulation [21.41643665626451]
  We propose an attack framework that extracts leakage of watermark patterns using a pre-trained vision model.<n>Unlike prior works requiring massive data or detector access, our method achieves both forgery and detection evasion with a single watermarked image.<n>Our work exposes the robustness-stealthiness paradox: current "robust" watermarks sacrifice security for distortion resistance, providing insights for future watermark design.
  arXiv  Detail & Related papers  (2025-02-10T12:55:08Z)
• WaterMAS: Sharpness-Aware Maximization for Neural Network Watermarking [11.717546811674884]
  WaterMAS is a substitutive, white-box neural network watermarking method that improves the trade-off among robustness, imperceptibility, and computational complexity. The robustness is ensured by limiting the attack's strength. The imperceptibility is ensured by inserting the watermark during the training process.
  arXiv  Detail & Related papers  (2024-09-05T20:22:01Z)
• Lazy Layers to Make Fine-Tuned Diffusion Models More Traceable [70.77600345240867]
  A novel arbitrary-in-arbitrary-out (AIAO) strategy makes watermarks resilient to fine-tuning-based removal. Unlike the existing methods of designing a backdoor for the input/output space of diffusion models, in our method, we propose to embed the backdoor into the feature space of sampled subpaths. Our empirical studies on the MS-COCO, AFHQ, LSUN, CUB-200, and DreamBooth datasets confirm the robustness of AIAO.
  arXiv  Detail & Related papers  (2024-05-01T12:03:39Z)
• Safe and Robust Watermark Injection with a Single OoD Image [90.71804273115585]
  Training a high-performance deep neural network requires large amounts of data and computational resources. We propose a safe and robust backdoor-based watermark injection technique. We induce random perturbation of model parameters during watermark injection to defend against common watermark removal attacks.
  arXiv  Detail & Related papers  (2023-09-04T19:58:35Z)
• Exploring Structure Consistency for Deep Model Watermarking [122.38456787761497]
  The intellectual property (IP) of Deep neural networks (DNNs) can be easily stolen'' by surrogate model attack. We propose a new watermarking methodology, namely structure consistency'', based on which a new deep structure-aligned model watermarking algorithm is designed.
  arXiv  Detail & Related papers  (2021-08-05T04:27:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.