PT-Mark: Invisible Watermarking for Text-to-image Diffusion Models via Semantic-aware Pivotal Tuning

• URL: http://arxiv.org/abs/2504.10853v2
• Date: Fri, 18 Apr 2025 04:58:10 GMT
• Title: PT-Mark: Invisible Watermarking for Text-to-image Diffusion Models via Semantic-aware Pivotal Tuning
• Authors: Yaopeng Wang, Huiyu Xu, Zhibo Wang, Jiacheng Du, Zhichao Li, Yiming Li, Qiu Wang, Kui Ren,
• Abstract summary: We present Semantic-aware Pivotal Tuning Watermarks (PT-Mark)<n>PT-Mark preserves both the semantics of diffusion images and the traceability of the watermark.<n>It achieves a 10% improvement in the performance of semantic preservation compared to state-of-the-art watermarking methods.
• Score: 19.170393134039568
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Watermarking for diffusion images has drawn considerable attention due to the widespread use of text-to-image diffusion models and the increasing need for their copyright protection. Recently, advanced watermarking techniques, such as Tree Ring, integrate watermarks by embedding traceable patterns (e.g., Rings) into the latent distribution during the diffusion process. Such methods disrupt the original semantics of the generated images due to the inevitable distribution shift caused by the watermarks, thereby limiting their practicality, particularly in digital art creation. In this work, we present Semantic-aware Pivotal Tuning Watermarks (PT-Mark), a novel invisible watermarking method that preserves both the semantics of diffusion images and the traceability of the watermark. PT-Mark preserves the original semantics of the watermarked image by gradually aligning the generation trajectory with the original (pivotal) trajectory while maintaining the traceable watermarks during whole diffusion denoising process. To achieve this, we first compute the salient regions of the watermark at each diffusion denoising step as a spatial prior to identify areas that can be aligned without disrupting the watermark pattern. Guided by the region, we then introduce an additional pivotal tuning branch that optimizes the text embedding to align the semantics while preserving the watermarks. Extensive evaluations demonstrate that PT-Mark can preserve the original semantics of the diffusion images while integrating robust watermarks. It achieves a 10% improvement in the performance of semantic preservation (i.e., SSIM, PSNR, and LPIPS) compared to state-of-the-art watermarking methods, while also showing comparable robustness against real-world perturbations and four times greater efficiency.

Related papers

• Forging and Removing Latent-Noise Diffusion Watermarks Using a Single Image [24.513881574366046]
  Previous watermarking schemes embed a secret key in the initial noise. We propose a black-box adversarial attack without presuming access to the diffusion model weights. We show that we can also apply a similar approach for watermark removal by learning perturbations to exit this region.
  arXiv  Detail & Related papers  (2025-04-27T18:29:26Z)
• Bridging Knowledge Gap Between Image Inpainting and Large-Area Visible Watermark Removal [57.84348166457113]
  We introduce a novel feature adapting framework that leverages the representation capacity of a pre-trained image inpainting model. Our approach bridges the knowledge gap between image inpainting and watermark removal by fusing information of the residual background content beneath watermarks into the inpainting backbone model. For relieving the dependence on high-quality watermark masks, we introduce a new training paradigm by utilizing coarse watermark masks to guide the inference process.
  arXiv  Detail & Related papers  (2025-04-07T02:37:14Z)
• SEAL: Semantic Aware Image Watermarking [26.606008778795193]
  We propose a novel watermarking method that embeds semantic information about the generated image directly into the watermark.<n>The key pattern can be inferred from the semantic embedding of the image using locality-sensitive hashing.<n>Our results suggest that content-aware watermarks can mitigate risks arising from image-generative models.
  arXiv  Detail & Related papers  (2025-03-15T15:29:05Z)
• SuperMark: Robust and Training-free Image Watermarking via Diffusion-based Super-Resolution [27.345134138673945]
  We propose SuperMark, a robust, training-free watermarking framework.<n>SuperMark embeds the watermark into initial Gaussian noise using existing techniques.<n>It then applies pre-trained Super-Resolution models to denoise the watermarked noise, producing the final watermarked image.<n>For extraction, the process is reversed: the watermarked image is inverted back to the initial watermarked noise via DDIM Inversion, from which the embedded watermark is extracted.<n>Experiments demonstrate that SuperMark achieves fidelity comparable to existing methods while significantly improving robustness.
  arXiv  Detail & Related papers  (2024-12-13T11:20:59Z)
• Shallow Diffuse: Robust and Invisible Watermarking through Low-Dimensional Subspaces in Diffusion Models [10.726987194250116]
  We introduce Shallow Diffuse, a new watermarking technique that embeds robust and invisible watermarks into diffusion model outputs. Our theoretical and empirical analyses show that Shallow Diffuse greatly enhances the consistency of data generation and the detectability of the watermark.
  arXiv  Detail & Related papers  (2024-10-28T14:51:04Z)
• Image Watermarks are Removable Using Controllable Regeneration from Clean Noise [26.09012436917272]
  A critical attribute of watermark techniques is their robustness against various manipulations.<n>We introduce a watermark removal approach capable of effectively nullifying state-of-the-art watermarking techniques.<n>Our methods offer superior visual consistency/quality and enhanced watermark removal performance compared to existing regeneration approaches.
  arXiv  Detail & Related papers  (2024-10-07T20:04:29Z)
• Removing Interference and Recovering Content Imaginatively for Visible Watermark Removal [63.576748565274706]
  This study introduces the Removing Interference and Recovering Content Imaginatively (RIRCI) framework. RIRCI embodies a two-stage approach: the initial phase centers on discerning and segregating the watermark component, while the subsequent phase focuses on background content restoration. To achieve meticulous background restoration, our proposed model employs a dual-path network capable of fully exploring the intrinsic background information beneath semi-transparent watermarks.
  arXiv  Detail & Related papers  (2023-12-22T02:19:23Z)
• FT-Shield: A Watermark Against Unauthorized Fine-tuning in Text-to-Image Diffusion Models [64.89896692649589]
  We propose FT-Shield, a watermarking solution tailored for the fine-tuning of text-to-image diffusion models. FT-Shield addresses copyright protection challenges by designing new watermark generation and detection strategies.
  arXiv  Detail & Related papers  (2023-10-03T19:50:08Z)
• WMFormer++: Nested Transformer for Visible Watermark Removal via Implict Joint Learning [68.00975867932331]
  Existing watermark removal methods mainly rely on UNet with task-specific decoder branches. We introduce an implicit joint learning paradigm to holistically integrate information from both branches. The results demonstrate our approach's remarkable superiority, surpassing existing state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2023-08-20T07:56:34Z)
• Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust [55.91987293510401]
  Watermarking the outputs of generative models is a crucial technique for tracing copyright and preventing potential harm from AI-generated content. We introduce a novel technique called Tree-Ring Watermarking that robustly fingerprints diffusion model outputs. Our watermark is semantically hidden in the image space and is far more robust than watermarking alternatives that are currently deployed.
  arXiv  Detail & Related papers  (2023-05-31T17:00:31Z)
• Watermarking Images in Self-Supervised Latent Spaces [75.99287942537138]
  We revisit watermarking techniques based on pre-trained deep networks, in the light of self-supervised approaches. We present a way to embed both marks and binary messages into their latent spaces, leveraging data augmentation at marking time.
  arXiv  Detail & Related papers  (2021-12-17T15:52:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.