Catch You Everything Everywhere: Guarding Textual Inversion via Concept Watermarking

• URL: http://arxiv.org/abs/2309.05940v1
• Date: Tue, 12 Sep 2023 03:33:13 GMT
• Title: Catch You Everything Everywhere: Guarding Textual Inversion via Concept Watermarking
• Authors: Weitao Feng, Jiyan He, Jie Zhang, Tianwei Zhang, Wenbo Zhou, Weiming Zhang, Nenghai Yu,
• Abstract summary: We propose the novel concept watermarking, where watermark information is embedded into the target concept and then extracted from generated images based on the watermarked concept. In practice, the concept owner can upload his concept with different watermarks (ie, serial numbers) to the platform, and the platform allocates different users with different serial numbers for subsequent tracing and forensics.
• Score: 67.60174799881597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AIGC (AI-Generated Content) has achieved tremendous success in many applications such as text-to-image tasks, where the model can generate high-quality images with diverse prompts, namely, different descriptions in natural languages. More surprisingly, the emerging personalization techniques even succeed in describing unseen concepts with only a few personal images as references, and there have been some commercial platforms for sharing the valuable personalized concept. However, such an advanced technique also introduces a severe threat, where malicious users can misuse the target concept to generate highly-realistic illegal images. Therefore, it becomes necessary for the platform to trace malicious users and hold them accountable. In this paper, we focus on guarding the most popular lightweight personalization model, ie, Textual Inversion (TI). To achieve it, we propose the novel concept watermarking, where watermark information is embedded into the target concept and then extracted from generated images based on the watermarked concept. Specifically, we jointly train a watermark encoder and a watermark decoder with the sampler in the loop. It shows great resilience to different diffusion sampling processes possibly chosen by malicious users, meanwhile preserving utility for normal use. In practice, the concept owner can upload his concept with different watermarks (ie, serial numbers) to the platform, and the platform allocates different users with different serial numbers for subsequent tracing and forensics.

Related papers

• Protect-Your-IP: Scalable Source-Tracing and Attribution against Personalized Generation [19.250673262185767]
  We propose a unified approach for image copyright source-tracing and attribution. We introduce an innovative watermarking-attribution method that blends proactive and passive strategies. We have conducted experiments using various celebrity portrait series sourced online.
  arXiv  Detail & Related papers  (2024-05-26T15:14:54Z)
• ProMark: Proactive Diffusion Watermarking for Causal Attribution [25.773438257321793]
  We propose ProMark, a causal attribution technique to attribute a synthetically generated image to its training data concepts. The concept information is proactively embedded into the input training images using imperceptible watermarks. We show that we can embed as many as $216$ unique watermarks into the training data, and each training image can contain more than one watermark.
  arXiv  Detail & Related papers  (2024-03-14T23:16:43Z)
• Visual Concept-driven Image Generation with Text-to-Image Diffusion Model [65.96212844602866]
  Text-to-image (TTI) models have demonstrated impressive results in generating high-resolution images of complex scenes. Recent approaches have extended these methods with personalization techniques that allow them to integrate user-illustrated concepts. However, the ability to generate images with multiple interacting concepts, such as human subjects, as well as concepts that may be entangled in one, or across multiple, image illustrations remains illusive. We propose a concept-driven TTI personalization framework that addresses these core challenges.
  arXiv  Detail & Related papers  (2024-02-18T07:28:37Z)
• RAW: A Robust and Agile Plug-and-Play Watermark Framework for AI-Generated Images with Provable Guarantees [33.61946642460661]
  This paper introduces a robust and agile watermark detection framework, dubbed as RAW. We employ a classifier that is jointly trained with the watermark to detect the presence of the watermark. We show that the framework provides provable guarantees regarding the false positive rate for misclassifying a watermarked image.
  arXiv  Detail & Related papers  (2024-01-23T22:00:49Z)
• FT-Shield: A Watermark Against Unauthorized Fine-tuning in Text-to-Image Diffusion Models [64.89896692649589]
  We propose FT-Shield, a watermarking solution tailored for the fine-tuning of text-to-image diffusion models. FT-Shield addresses copyright protection challenges by designing new watermark generation and detection strategies.
  arXiv  Detail & Related papers  (2023-10-03T19:50:08Z)
• Break-A-Scene: Extracting Multiple Concepts from a Single Image [80.47666266017207]
  We introduce the task of textual scene decomposition. We propose augmenting the input image with masks that indicate the presence of target concepts. We then present a novel two-phase customization process.
  arXiv  Detail & Related papers  (2023-05-25T17:59:04Z)
• Watermarking Images in Self-Supervised Latent Spaces [75.99287942537138]
  We revisit watermarking techniques based on pre-trained deep networks, in the light of self-supervised approaches. We present a way to embed both marks and binary messages into their latent spaces, leveraging data augmentation at marking time.
  arXiv  Detail & Related papers  (2021-12-17T15:52:46Z)
• Split then Refine: Stacked Attention-guided ResUNets for Blind Single Image Visible Watermark Removal [69.92767260794628]
  Previous watermark removal methods require to gain the watermark location from users or train a multi-task network to recover the background indiscriminately. We propose a novel two-stage framework with a stacked attention-guided ResUNets to simulate the process of detection, removal and refinement. We extensively evaluate our algorithm over four different datasets under various settings and the experiments show that our approach outperforms other state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2020-12-13T09:05:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.