Dark Miner: Defend against undesired generation for text-to-image diffusion models

• URL: http://arxiv.org/abs/2409.17682v2
• Date: Mon, 25 Nov 2024 13:31:05 GMT
• Title: Dark Miner: Defend against undesired generation for text-to-image diffusion models
• Authors: Zheling Meng, Bo Peng, Xiaochuan Jin, Yue Jiang, Jing Dong, Wei Wang,
• Abstract summary: We analyze the erasure task and point out that existing methods cannot guarantee the minimization of the total probabilities of undesired generation. We propose Dark Miner, a recurring three-stage process that comprises mining, verifying, and circumventing. Compared with the previous methods, our method achieves better erasure and defense results, especially under multiple adversarial attacks.
• Score: 13.86760397597925
• License:
• Abstract: Text-to-image diffusion models have been demonstrated with undesired generation due to unfiltered large-scale training data, such as sexual images and copyrights, necessitating the erasure of undesired concepts. Most existing methods focus on modifying the generation probabilities conditioned on the texts containing target concepts. However, they fail to guarantee the desired generation of texts unseen in the training phase, especially for the adversarial texts from malicious attacks. In this paper, we analyze the erasure task and point out that existing methods cannot guarantee the minimization of the total probabilities of undesired generation. To tackle this problem, we propose Dark Miner. It entails a recurring three-stage process that comprises mining, verifying, and circumventing. This method greedily mines embeddings with maximum generation probabilities of target concepts and more effectively reduces their generation. In the experiments, we evaluate its performance on the inappropriateness, object, and style concepts. Compared with the previous methods, our method achieves better erasure and defense results, especially under multiple adversarial attacks, while preserving the native generation capability of the models. Our code will be available at https://github.com/RichardSunnyMeng/DarkMiner-offical-codes.

Related papers

• Safe Text-to-Image Generation: Simply Sanitize the Prompt Embedding [13.481343482138888]
  We propose a vision-agnostic safe generation framework, Embedding Sanitizer (ES) ES focuses on erasing inappropriate concepts from prompt embeddings and uses the sanitized embeddings to guide the model for safe generation. ES significantly outperforms existing safeguards in terms of interpretability and controllability while maintaining generation quality.
  arXiv  Detail & Related papers  (2024-11-15T16:29:02Z)
• Chain-of-Jailbreak Attack for Image Generation Models via Editing Step by Step [62.82566977845765]
  We introduce a novel jailbreaking method called Chain-of-Jailbreak (CoJ) attack, which compromises image generation models through a step-by-step editing process. Our CoJ attack method can successfully bypass the safeguards of models for over 60% cases. We also propose an effective prompting-based method, Think Twice Prompting, that can successfully defend over 95% of CoJ attack.
  arXiv  Detail & Related papers  (2024-10-04T19:04:43Z)
• Avoiding Generative Model Writer's Block With Embedding Nudging [8.3196702956302]
  We focus on the latent diffusion image generative models and how one can prevent them to generate particular images while generating similar images with limited overhead. Our method successfully prevents the generation of memorized training images while maintaining comparable image quality and relevance to the unmodified model.
  arXiv  Detail & Related papers  (2024-08-28T00:07:51Z)
• Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Models [76.39651111467832]
  We introduce Reliable and Efficient Concept Erasure (RECE), a novel approach that modifies the model in 3 seconds without necessitating additional fine-tuning. To mitigate inappropriate content potentially represented by derived embeddings, RECE aligns them with harmless concepts in cross-attention layers. The derivation and erasure of new representation embeddings are conducted iteratively to achieve a thorough erasure of inappropriate concepts.
  arXiv  Detail & Related papers  (2024-07-17T08:04:28Z)
• Six-CD: Benchmarking Concept Removals for Benign Text-to-image Diffusion Models [58.74606272936636]
  Text-to-image (T2I) diffusion models have shown exceptional capabilities in generating images that closely correspond to textual prompts. The models could be exploited for malicious purposes, such as generating images with violence or nudity, or creating unauthorized portraits of public figures in inappropriate contexts. concept removal methods have been proposed to modify diffusion models to prevent the generation of malicious and unwanted concepts.
  arXiv  Detail & Related papers  (2024-06-21T03:58:44Z)
• Latent Guard: a Safety Framework for Text-to-image Generation [64.49596711025993]
  Existing safety measures are either based on text blacklists, which can be easily circumvented, or harmful content classification. We propose Latent Guard, a framework designed to improve safety measures in text-to-image generation. Inspired by blacklist-based approaches, Latent Guard learns a latent space on top of the T2I model's text encoder, where it is possible to check the presence of harmful concepts.
  arXiv  Detail & Related papers  (2024-04-11T17:59:52Z)
• Data Redaction from Conditional Generative Models [38.479256505860825]
  We study how to post-edit an already-trained conditional generative model so that it redacts certain conditionals that will, with high probability, lead to undesirable content. We conduct experiments on redacting prompts in text-to-image models and redacting voices in text-to-speech models.
  arXiv  Detail & Related papers  (2023-05-18T23:58:53Z)
• Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models [79.50701155336198]
  textbfForget-Me-Not is designed to safely remove specified IDs, objects, or styles from a well-configured text-to-image model in as little as 30 seconds. We demonstrate that Forget-Me-Not can effectively eliminate targeted concepts while maintaining the model's performance on other concepts. It can also be adapted as a lightweight model patch for Stable Diffusion, allowing for concept manipulation and convenient distribution.
  arXiv  Detail & Related papers  (2023-03-30T17:58:11Z)
• Diffusion Models for Adversarial Purification [69.1882221038846]
  Adrial purification refers to a class of defense methods that remove adversarial perturbations using a generative model. We propose DiffPure that uses diffusion models for adversarial purification. Our method achieves the state-of-the-art results, outperforming current adversarial training and adversarial purification methods.
  arXiv  Detail & Related papers  (2022-05-16T06:03:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.