Imperceptible Protection against Style Imitation from Diffusion Models

• URL: http://arxiv.org/abs/2403.19254v2
• Date: Wed, 28 Aug 2024 15:13:45 GMT
• Title: Imperceptible Protection against Style Imitation from Diffusion Models
• Authors: Namhyuk Ahn, Wonhyuk Ahn, KiYoon Yoo, Daesik Kim, Seung-Hun Nam,
• Abstract summary: We introduce a visually improved protection method while preserving its protection capability. We devise a perceptual map to highlight areas sensitive to human eyes, guided by instance-aware refinement. We also introduce a difficulty-aware protection by predicting how difficult the artwork is to protect and dynamically adjusting the intensity.
• Score: 9.548195579003897
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent progress in diffusion models has profoundly enhanced the fidelity of image generation, but it has raised concerns about copyright infringements. While prior methods have introduced adversarial perturbations to prevent style imitation, most are accompanied by the degradation of artworks' visual quality. Recognizing the importance of maintaining this, we introduce a visually improved protection method while preserving its protection capability. To this end, we devise a perceptual map to highlight areas sensitive to human eyes, guided by instance-aware refinement, which refines the protection intensity accordingly. We also introduce a difficulty-aware protection by predicting how difficult the artwork is to protect and dynamically adjusting the intensity based on this. Lastly, we integrate a perceptual constraints bank to further improve the imperceptibility. Results show that our method substantially elevates the quality of the protected image without compromising on protection efficacy.

Related papers

• DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing [93.45507533317405]
  DiffusionGuard is a robust and effective defense method against unauthorized edits by diffusion-based image editing models. We introduce a novel objective that generates adversarial noise targeting the early stage of the diffusion process. We also introduce a mask-augmentation technique to enhance robustness against various masks during test time.
  arXiv  Detail & Related papers  (2024-10-08T05:19:19Z)
• Pixel Is Not A Barrier: An Effective Evasion Attack for Pixel-Domain Diffusion Models [9.905296922309157]
  Diffusion Models have emerged as powerful generative models for high-quality image synthesis, with many subsequent image editing techniques based on them. Previous works have attempted to safeguard images from diffusion-based editing by adding imperceptible perturbations. Our work proposes a novel attacking framework with a feature representation attack loss that exploits vulnerabilities in denoising UNets and a latent optimization strategy to enhance the naturalness of protected images.
  arXiv  Detail & Related papers  (2024-08-21T17:56:34Z)
• Visual-Friendly Concept Protection via Selective Adversarial Perturbations [23.780603071185197]
  We propose the Visual-Friendly Concept Protection (VCPro) framework, which prioritizes the protection of key concepts chosen by the image owner. To ensure these perturbations are as inconspicuous as possible, we introduce a relaxed optimization objective. Experiments validate that VCPro achieves a better trade-off between the visibility of perturbations and protection effectiveness.
  arXiv  Detail & Related papers  (2024-08-16T04:14:28Z)
• Rethinking and Defending Protective Perturbation in Personalized Diffusion Models [21.30373461975769]
  We study the fine-tuning process of personalized diffusion models (PDMs) through the lens of shortcut learning. PDMs are susceptible to minor adversarial perturbations, leading to significant degradation when fine-tuned on corrupted datasets. We propose a systematic defense framework that includes data purification and contrastive decoupling learning.
  arXiv  Detail & Related papers  (2024-06-27T07:14:14Z)
• EnTruth: Enhancing the Traceability of Unauthorized Dataset Usage in Text-to-image Diffusion Models with Minimal and Robust Alterations [73.94175015918059]
  We introduce a novel approach, EnTruth, which Enhances Traceability of unauthorized dataset usage. By strategically incorporating the template memorization, EnTruth can trigger the specific behavior in unauthorized models as the evidence of infringement. Our method is the first to investigate the positive application of memorization and use it for copyright protection, which turns a curse into a blessing.
  arXiv  Detail & Related papers  (2024-06-20T02:02:44Z)
• Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI [61.35083814817094]
  Several protection tools against style mimicry have been developed that incorporate small adversarial perturbations into artworks published online. We find that low-effort and "off-the-shelf" techniques, such as image upscaling, are sufficient to create robust mimicry methods that significantly degrade existing protections. We caution that tools based on adversarial perturbations cannot reliably protect artists from the misuse of generative AI.
  arXiv  Detail & Related papers  (2024-06-17T18:51:45Z)
• Diffusion-Based Hierarchical Image Steganography [60.69791384893602]
  Hierarchical Image Steganography is a novel method that enhances the security and capacity of embedding multiple images into a single container. It exploits the robustness of the Diffusion Model alongside the reversibility of the Flow Model. The innovative structure can autonomously generate a container image, thereby securely and efficiently concealing multiple images and text.
  arXiv  Detail & Related papers  (2024-05-19T11:29:52Z)
• Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model [61.53213964333474]
  We propose a unified framework Adv-Diffusion that can generate imperceptible adversarial identity perturbations in the latent space but not the raw pixel space. Specifically, we propose the identity-sensitive conditioned diffusion generative model to generate semantic perturbations in the surroundings. The designed adaptive strength-based adversarial perturbation algorithm can ensure both attack transferability and stealthiness.
  arXiv  Detail & Related papers  (2023-12-18T15:25:23Z)
• IRAD: Implicit Representation-driven Image Resampling against Adversarial Attacks [16.577595936609665]
  We introduce a novel approach to counter adversarial attacks, namely, image resampling. Image resampling transforms a discrete image into a new one, simulating the process of scene recapturing or rerendering as specified by a geometrical transformation. We show that our method significantly enhances the adversarial robustness of diverse deep models against various attacks while maintaining high accuracy on clean images.
  arXiv  Detail & Related papers  (2023-10-18T11:19:32Z)
• Adv-Attribute: Inconspicuous and Transferable Adversarial Attack on Face Recognition [111.1952945740271]
  Adversarial Attributes (Adv-Attribute) is designed to generate inconspicuous and transferable attacks on face recognition. Experiments on the FFHQ and CelebA-HQ datasets show that the proposed Adv-Attribute method achieves the state-of-the-art attacking success rates.
  arXiv  Detail & Related papers  (2022-10-13T09:56:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.