DiffGuard: Text-Based Safety Checker for Diffusion Models

• URL: http://arxiv.org/abs/2412.00064v2
• Date: Wed, 19 Feb 2025 15:51:43 GMT
• Title: DiffGuard: Text-Based Safety Checker for Diffusion Models
• Abstract summary: This paper presents a novel text-based safety filter that outperforms existing solutions.<n>Our research is driven by the critical need to address the misuse of AI-generated content, especially in the context of information warfare.
• Score: 1.4375779579519912
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in Diffusion Models have enabled the generation of images from text, with powerful closed-source models like DALL-E and Midjourney leading the way. However, open-source alternatives, such as StabilityAI's Stable Diffusion, offer comparable capabilities. These open-source models, hosted on Hugging Face, come equipped with ethical filter protections designed to prevent the generation of explicit images. This paper reveals first their limitations and then presents a novel text-based safety filter that outperforms existing solutions. Our research is driven by the critical need to address the misuse of AI-generated content, especially in the context of information warfare. DiffGuard enhances filtering efficacy, achieving a performance that surpasses the best existing filters by over 14%.

Related papers

• DiSCO: Defending text-to-image generation through distribution-guided contrastive prompt optimization [65.89030524420933]
  DiSCO is a zero-shot, strictly black-box defense that operates entirely at the prompt level as a plug-and-play module.<n>We demonstrate that DiSCO consistently enhances the safety of both undefended and defended models.<n>As a black-box, architecture-agnostic module, DiSCO can be readily applied to any text-to-image system.
  arXiv  Detail & Related papers  (2026-08-17T19:13:55Z)
• The Safety-Aware Denoiser for Text Diffusion Models [1.6800992804614623]
  We propose the Safety-Aware Denoiser (SAD), a safety-guidance framework in text diffusion models.<n>SAD modifies the iterative denoising process such that the text sample at the final denoising step is steered toward provably safe regions.<n>We evaluate the safety of the generated text using the SAD, with respect to hazard taxonomy, memorization, and jailbreak.
  arXiv  Detail & Related papers  (2026-04-28T00:44:30Z)
• WaterVIB: Learning Minimal Sufficient Watermark Representations via Variational Information Bottleneck [54.83612291453286]
  Existing methods fail because they entangle the watermark with high-frequency cover texture, which is susceptible to being rewritten during generative purification.<n>We propose WaterVIB, a framework that reformulates the encoder as an information sieve via the Variational Information Bottleneck.<n>This effectively filters out redundant cover nuances prone to generative shifts, retaining only the essential signal invariant to regeneration.
  arXiv  Detail & Related papers  (2026-02-25T02:38:17Z)
• Responsible Diffusion Models via Constraining Text Embeddings within Safe Regions [35.28819408507869]
  Concerns have also arisen regarding their potential to produce Not Safe for Work (NSFW) content and exhibit social biases.<n>We propose a novel self-discovery approach to identify a semantic direction vector in the embedding space to restrict text embedding within a safe region.<n>Our method can effectively reduce NSFW content and social bias generated by diffusion models compared to several state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-05-21T12:10:26Z)
• CROPS: Model-Agnostic Training-Free Framework for Safe Image Synthesis with Latent Diffusion Models [13.799517170191919]
  Recent research has shown that safety checkers have vulnerabilities against adversarial attacks, allowing them to generate Not Safe For Work (NSFW) images. We propose CROPS, a model-agnostic framework that easily defends against adversarial attacks generating NSFW images without requiring additional training.
  arXiv  Detail & Related papers  (2025-01-09T16:43:21Z)
• DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing [93.45507533317405]
  DiffusionGuard is a robust and effective defense method against unauthorized edits by diffusion-based image editing models. We introduce a novel objective that generates adversarial noise targeting the early stage of the diffusion process. We also introduce a mask-augmentation technique to enhance robustness against various masks during test time.
  arXiv  Detail & Related papers  (2024-10-08T05:19:19Z)
• Pixel Is Not A Barrier: An Effective Evasion Attack for Pixel-Domain Diffusion Models [9.905296922309157]
  Diffusion Models have emerged as powerful generative models for high-quality image synthesis, with many subsequent image editing techniques based on them. Previous works have attempted to safeguard images from diffusion-based editing by adding imperceptible perturbations. Our work proposes a novel attacking framework with a feature representation attack loss that exploits vulnerabilities in denoising UNets and a latent optimization strategy to enhance the naturalness of protected images.
  arXiv  Detail & Related papers  (2024-08-21T17:56:34Z)
• Direct Unlearning Optimization for Robust and Safe Text-to-Image Models [29.866192834825572]
  Unlearning techniques have been developed to remove the model's ability to generate potentially harmful content. These methods are easily bypassed by adversarial attacks, making them unreliable for ensuring the safety of generated images. We propose Direct Unlearning Optimization (DUO), a novel framework for removing Not Safe For Work (NSFW) content from T2I models.
  arXiv  Detail & Related papers  (2024-07-17T08:19:11Z)
• EnTruth: Enhancing the Traceability of Unauthorized Dataset Usage in Text-to-image Diffusion Models with Minimal and Robust Alterations [73.94175015918059]
  We introduce a novel approach, EnTruth, which Enhances Traceability of unauthorized dataset usage. By strategically incorporating the template memorization, EnTruth can trigger the specific behavior in unauthorized models as the evidence of infringement. Our method is the first to investigate the positive application of memorization and use it for copyright protection, which turns a curse into a blessing.
  arXiv  Detail & Related papers  (2024-06-20T02:02:44Z)
• Watch the Watcher! Backdoor Attacks on Security-Enhancing Diffusion Models [65.30406788716104]
  This work investigates the vulnerabilities of security-enhancing diffusion models. We demonstrate that these models are highly susceptible to DIFF2, a simple yet effective backdoor attack. Case studies show that DIFF2 can significantly reduce both post-purification and certified accuracy across benchmark datasets and models.
  arXiv  Detail & Related papers  (2024-06-14T02:39:43Z)
• Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model [61.53213964333474]
  We propose a unified framework Adv-Diffusion that can generate imperceptible adversarial identity perturbations in the latent space but not the raw pixel space. Specifically, we propose the identity-sensitive conditioned diffusion generative model to generate semantic perturbations in the surroundings. The designed adaptive strength-based adversarial perturbation algorithm can ensure both attack transferability and stealthiness.
  arXiv  Detail & Related papers  (2023-12-18T15:25:23Z)
• Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts [63.61248884015162]
  Text-to-image diffusion models have shown remarkable ability in high-quality content generation. This work proposes Prompting4 Debugging (P4D) as a tool that automatically finds problematic prompts for diffusion models. Our result shows that around half of prompts in existing safe prompting benchmarks which were originally considered "safe" can actually be manipulated to bypass many deployed safety mechanisms.
  arXiv  Detail & Related papers  (2023-09-12T11:19:36Z)
• FLIRT: Feedback Loop In-context Red Teaming [79.63896510559357]
  We propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities. Our framework uses in-context learning in a feedback loop to red team models and trigger them into unsafe content generation.
  arXiv  Detail & Related papers  (2023-08-08T14:03:08Z)
• Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation [25.55296442023984]
  We propose a method, Unlearnable Diffusion Perturbation, to safeguard images from unauthorized exploitation. This achievement holds significant importance in real-world scenarios, as it contributes to the protection of privacy and copyright against AI-generated content.
  arXiv  Detail & Related papers  (2023-06-02T20:19:19Z)
• CRoSS: Diffusion Model Makes Controllable, Robust and Secure Image Steganography [15.705627450233504]
  We propose a novel image steganography framework, named Controllable, Robust and Secure Image Steganography (CRoSS) CRoSS has significant advantages in controllability, robustness, and security compared to cover-based image steganography methods.
  arXiv  Detail & Related papers  (2023-05-26T13:52:57Z)
• DiffProtect: Generate Adversarial Examples with Diffusion Models for Facial Privacy Protection [64.77548539959501]
  DiffProtect produces more natural-looking encrypted images than state-of-the-art methods. It achieves significantly higher attack success rates, e.g., 24.5% and 25.1% absolute improvements on the CelebA-HQ and FFHQ datasets.
  arXiv  Detail & Related papers  (2023-05-23T02:45:49Z)
• SneakyPrompt: Jailbreaking Text-to-image Generative Models [20.645304189835944]
  We propose SneakyPrompt, the first automated attack framework, to jailbreak text-to-image generative models. Given a prompt that is blocked by a safety filter, SneakyPrompt repeatedly queries the text-to-image generative model and strategically perturbs tokens in the prompt based on the query results to bypass the safety filter. Our evaluation shows that SneakyPrompt not only successfully generates NSFW images, but also outperforms existing text adversarial attacks when extended to jailbreak text-to-image generative models.
  arXiv  Detail & Related papers  (2023-05-20T03:41:45Z)
• Red-Teaming the Stable Diffusion Safety Filter [5.683172456953383]
  We show that it is easy to generate disturbing content that bypasses the safety filter. We argue safety measures in future model releases should strive to be fully open and properly documented.
  arXiv  Detail & Related papers  (2022-10-03T14:04:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.