When Image Generation Goes Wrong: A Safety Analysis of Stable Diffusion Models

• URL: http://arxiv.org/abs/2411.15516v1
• Date: Sat, 23 Nov 2024 10:42:43 GMT
• Title: When Image Generation Goes Wrong: A Safety Analysis of Stable Diffusion Models
• Authors: Matthias Schneider, Thilo Hagendorff,
• Abstract summary: This study investigates the ability of ten popular Stable Diffusion models to generate harmful images. We demonstrate that these models respond to harmful prompts by generating inappropriate content. Our findings demonstrate a complete lack of any refusal behavior or safety measures in the models observed.
• Score: 0.0
• License:
• Abstract: Text-to-image models are increasingly popular and impactful, yet concerns regarding their safety and fairness remain. This study investigates the ability of ten popular Stable Diffusion models to generate harmful images, including NSFW, violent, and personally sensitive material. We demonstrate that these models respond to harmful prompts by generating inappropriate content, which frequently displays troubling biases, such as the disproportionate portrayal of Black individuals in violent contexts. Our findings demonstrate a complete lack of any refusal behavior or safety measures in the models observed. We emphasize the importance of addressing this issue as image generation technologies continue to become more accessible and incorporated into everyday applications.

Related papers

• Anonymization Prompt Learning for Facial Privacy-Preserving Text-to-Image Generation [56.46932751058042]
  We train a learnable prompt prefix for text-to-image diffusion models, which forces the model to generate anonymized facial identities. Experiments demonstrate the successful anonymization performance of APL, which anonymizes any specific individuals without compromising the quality of non-identity-specific image generation.
  arXiv  Detail & Related papers  (2024-05-27T07:38:26Z)
• ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Users [18.3621509910395]
  We propose a novel Automatic Red-Teaming framework, ART, to evaluate the safety risks of text-to-image models. With our comprehensive experiments, we reveal the toxicity of the popular open-source text-to-image models. We also introduce three large-scale red-teaming datasets for studying the safety risks associated with text-to-image models.
  arXiv  Detail & Related papers  (2024-05-24T07:44:27Z)
• On the Proactive Generation of Unsafe Images From Text-To-Image Models Using Benign Prompts [38.63253101205306]
  Previous studies have successfully demonstrated that manipulated prompts can elicit text-to-image models to generate unsafe images. We propose two poisoning attacks: a basic attack and a utility-preserving attack. Our findings underscore the potential risks of adopting text-to-image models in real-world scenarios.
  arXiv  Detail & Related papers  (2023-10-25T13:10:44Z)
• SurrogatePrompt: Bypassing the Safety Filter of Text-to-Image Models via Substitution [21.93748586123046]
  We develop and exhibit the first prompt attacks on Midjourney, resulting in the production of abundant NSFW images. Our framework, SurrogatePrompt, systematically generates attack prompts, utilizing large language models, image-to-text, and image-to-image modules. Results disclose an 88% success rate in bypassing Midjourney's proprietary safety filter with our attack prompts.
  arXiv  Detail & Related papers  (2023-09-25T13:20:15Z)
• Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts [63.61248884015162]
  Text-to-image diffusion models have shown remarkable ability in high-quality content generation. This work proposes Prompting4 Debugging (P4D) as a tool that automatically finds problematic prompts for diffusion models. Our result shows that around half of prompts in existing safe prompting benchmarks which were originally considered "safe" can actually be manipulated to bypass many deployed safety mechanisms.
  arXiv  Detail & Related papers  (2023-09-12T11:19:36Z)
• FLIRT: Feedback Loop In-context Red Teaming [79.63896510559357]
  We propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities. Our framework uses in-context learning in a feedback loop to red team models and trigger them into unsafe content generation.
  arXiv  Detail & Related papers  (2023-08-08T14:03:08Z)
• Towards Safe Self-Distillation of Internet-Scale Text-to-Image Diffusion Models [63.20512617502273]
  We propose a method called SDD to prevent problematic content generation in text-to-image diffusion models. Our method eliminates a much greater proportion of harmful content from the generated images without degrading the overall image quality.
  arXiv  Detail & Related papers  (2023-07-12T07:48:29Z)
• Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models [103.61066310897928]
  Recent text-to-image generative models have demonstrated an unparalleled ability to generate diverse and creative imagery guided by a target text prompt. While revolutionary, current state-of-the-art diffusion models may still fail in generating images that fully convey the semantics in the given text prompt. We analyze the publicly available Stable Diffusion model and assess the existence of catastrophic neglect, where the model fails to generate one or more of the subjects from the input prompt. We introduce the concept of Generative Semantic Nursing (GSN), where we seek to intervene in the generative process on the fly during inference time to improve the faithfulness
  arXiv  Detail & Related papers  (2023-01-31T18:10:38Z)
• Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models [18.701950647429]
  Text-conditioned image generation models suffer from degenerated and biased human behavior. We present safe latent diffusion (SLD) to help combat these undesired side effects. We show that SLD removes and suppresses inappropriate image parts during the diffusion process.
  arXiv  Detail & Related papers  (2022-11-09T18:54:25Z)
• Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale [61.555788332182395]
  We investigate the potential for machine learning models to amplify dangerous and complex stereotypes. We find a broad range of ordinary prompts produce stereotypes, including prompts simply mentioning traits, descriptors, occupations, or objects.
  arXiv  Detail & Related papers  (2022-11-07T18:31:07Z)
• Membership Inference Attacks Against Text-to-image Generation Models [23.39695974954703]
  This paper performs the first privacy analysis of text-to-image generation models through the lens of membership inference. We propose three key intuitions about membership information and design four attack methodologies accordingly. All of the proposed attacks can achieve significant performance, in some cases even close to an accuracy of 1, and thus the corresponding risk is much more severe than that shown by existing membership inference attacks.
  arXiv  Detail & Related papers  (2022-10-03T14:31:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.