BSPA: Exploring Black-box Stealthy Prompt Attacks against Image Generators

• URL: http://arxiv.org/abs/2402.15218v1
• Date: Fri, 23 Feb 2024 09:28:16 GMT
• Title: BSPA: Exploring Black-box Stealthy Prompt Attacks against Image Generators
• Authors: Yu Tian, Xiao Yang, Yinpeng Dong, Heming Yang, Hang Su, Jun Zhu
• Abstract summary: Large image generators offer significant transformative potential across diverse sectors. Some studies reveal that image generators are notably susceptible to attacks and generate Not Suitable For Work (NSFW) contents. We introduce a black-box stealthy prompt attack that adopts a retriever to simulate attacks from API users.
• Score: 43.23698370787517
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Extremely large image generators offer significant transformative potential across diverse sectors. It allows users to design specific prompts to generate realistic images through some black-box APIs. However, some studies reveal that image generators are notably susceptible to attacks and generate Not Suitable For Work (NSFW) contents by manually designed toxin texts, especially imperceptible to human observers. We urgently need a multitude of universal and transferable prompts to improve the safety of image generators, especially black-box-released APIs. Nevertheless, they are constrained by labor-intensive design processes and heavily reliant on the quality of the given instructions. To achieve this, we introduce a black-box stealthy prompt attack (BSPA) that adopts a retriever to simulate attacks from API users. It can effectively harness filter scores to tune the retrieval space of sensitive words for matching the input prompts, thereby crafting stealthy prompts tailored for image generators. Significantly, this approach is model-agnostic and requires no internal access to the model's features, ensuring its applicability to a wide range of image generators. Building on BSPA, we have constructed an automated prompt tool and a comprehensive prompt attack dataset (NSFWeval). Extensive experiments demonstrate that BSPA effectively explores the security vulnerabilities in a variety of state-of-the-art available black-box models, including Stable Diffusion XL, Midjourney, and DALL-E 2/3. Furthermore, we develop a resilient text filter and offer targeted recommendations to ensure the security of image generators against prompt attacks in the future.

Related papers

• GenBreak: Red Teaming Text-to-Image Generators Using Large Language Models [65.91565607573786]
  Text-to-image (T2I) models can be misused to generate harmful content, including nudity or violence.<n>Recent research on red-teaming and adversarial attacks against T2I models has notable limitations.<n>We propose GenBreak, a framework that fine-tunes a red-team large language model (LLM) to systematically explore underlying vulnerabilities.
  arXiv  Detail & Related papers  (2025-06-11T09:09:12Z)
• Transferable Adversarial Attacks on Black-Box Vision-Language Models [63.22532779621001]
  adversarial attacks can transfer from open-source to proprietary black-box models in text-only and vision-only contexts.<n>We show that attackers can craft perturbations to induce specific attacker-chosen interpretations of visual information.<n>We discover that universal perturbations -- modifications applicable to a wide set of images -- can consistently induce these misinterpretations.
  arXiv  Detail & Related papers  (2025-05-02T06:51:11Z)
• Towards Safe Synthetic Image Generation On the Web: A Multimodal Robust NSFW Defense and Million Scale Dataset [20.758637391023345]
  A multimodal defense is developed to distinguish safe and NSFW text and images. Our model performs well against existing SOTA NSFW detection methods in terms of accuracy and recall.
  arXiv  Detail & Related papers  (2025-04-16T02:10:42Z)
• Distorting Embedding Space for Safety: A Defense Mechanism for Adversarially Robust Diffusion Models [4.5656369638728656]
  Distorting Embedding Space (DES) is a text encoder-based defense mechanism. DES transforms unsafe embeddings, extracted from a text encoder using unsafe prompts, toward carefully calculated safe embedding regions. DES also neutralizes the nudity embedding, extracted using prompt nudity", by aligning it with neutral embedding to enhance robustness against adversarial attacks.
  arXiv  Detail & Related papers  (2025-01-31T04:14:05Z)
• CROPS: Model-Agnostic Training-Free Framework for Safe Image Synthesis with Latent Diffusion Models [13.799517170191919]
  Recent research has shown that safety checkers have vulnerabilities against adversarial attacks, allowing them to generate Not Safe For Work (NSFW) images. We propose CROPS, a model-agnostic framework that easily defends against adversarial attacks generating NSFW images without requiring additional training.
  arXiv  Detail & Related papers  (2025-01-09T16:43:21Z)
• AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion models [20.37481116837779]
  AdvI2I is a novel framework that manipulates input images to induce diffusion models to generate NSFW content. By optimizing a generator to craft adversarial images, AdvI2I circumvents existing defense mechanisms. We show that both AdvI2I and AdvI2I-Adaptive can effectively bypass current safeguards.
  arXiv  Detail & Related papers  (2024-10-28T19:15:06Z)
• Chain-of-Jailbreak Attack for Image Generation Models via Editing Step by Step [62.82566977845765]
  We introduce a novel jailbreaking method called Chain-of-Jailbreak (CoJ) attack, which compromises image generation models through a step-by-step editing process. Our CoJ attack method can successfully bypass the safeguards of models for over 60% cases. We also propose an effective prompting-based method, Think Twice Prompting, that can successfully defend over 95% of CoJ attack.
  arXiv  Detail & Related papers  (2024-10-04T19:04:43Z)
• Multimodal Pragmatic Jailbreak on Text-to-image Models [43.67831238116829]
  This work introduces a novel type of jailbreak, which triggers T2I models to generate the image with visual text. We benchmark nine representative T2I models, including two close-source commercial models. All tested models suffer from such type of jailbreak, with rates of unsafe generation ranging from 8% to 74%.
  arXiv  Detail & Related papers  (2024-09-27T21:23:46Z)
• RT-Attack: Jailbreaking Text-to-Image Models via Random Token [24.61198605177661]
  We introduce a two-stage query-based black-box attack method utilizing random search. In the first stage, we establish a preliminary prompt by maximizing the semantic similarity between the adversarial and target harmful prompts. In the second stage, we use this initial prompt to refine our approach, creating a detailed adversarial prompt aimed at jailbreaking.
  arXiv  Detail & Related papers  (2024-08-25T17:33:40Z)
• EIUP: A Training-Free Approach to Erase Non-Compliant Concepts Conditioned on Implicit Unsafe Prompts [32.590822043053734]
  Non-toxic text still carries a risk of generating non-compliant images, which is referred to as implicit unsafe prompts. We propose a simple yet effective approach that incorporates non-compliant concepts into an erasure prompt. Our method exhibits superior erasure effectiveness while achieving high scores in image fidelity.
  arXiv  Detail & Related papers  (2024-08-02T05:17:14Z)
• Latent Guard: a Safety Framework for Text-to-image Generation [64.49596711025993]
  Existing safety measures are either based on text blacklists, which can be easily circumvented, or harmful content classification. We propose Latent Guard, a framework designed to improve safety measures in text-to-image generation. Inspired by blacklist-based approaches, Latent Guard learns a latent space on top of the T2I model's text encoder, where it is possible to check the presence of harmful concepts.
  arXiv  Detail & Related papers  (2024-04-11T17:59:52Z)
• SurrogatePrompt: Bypassing the Safety Filter of Text-to-Image Models via Substitution [21.93748586123046]
  We develop and exhibit the first prompt attacks on Midjourney, resulting in the production of abundant NSFW images. Our framework, SurrogatePrompt, systematically generates attack prompts, utilizing large language models, image-to-text, and image-to-image modules. Results disclose an 88% success rate in bypassing Midjourney's proprietary safety filter with our attack prompts.
  arXiv  Detail & Related papers  (2023-09-25T13:20:15Z)
• Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts [63.61248884015162]
  Text-to-image diffusion models have shown remarkable ability in high-quality content generation. This work proposes Prompting4 Debugging (P4D) as a tool that automatically finds problematic prompts for diffusion models. Our result shows that around half of prompts in existing safe prompting benchmarks which were originally considered "safe" can actually be manipulated to bypass many deployed safety mechanisms.
  arXiv  Detail & Related papers  (2023-09-12T11:19:36Z)
• BAGM: A Backdoor Attack for Manipulating Text-to-Image Generative Models [54.19289900203071]
  The rise in popularity of text-to-image generative artificial intelligence has attracted widespread public interest. We demonstrate that this technology can be attacked to generate content that subtly manipulates its users. We propose a Backdoor Attack on text-to-image Generative Models (BAGM) Our attack is the first to target three popular text-to-image generative models across three stages of the generative process.
  arXiv  Detail & Related papers  (2023-07-31T08:34:24Z)
• Mask and Restore: Blind Backdoor Defense at Test Time with Masked Autoencoder [57.739693628523]
  We propose a framework for blind backdoor defense with Masked AutoEncoder (BDMAE) BDMAE detects possible triggers in the token space using image structural similarity and label consistency between the test image and MAE restorations. Our approach is blind to the model restorations, trigger patterns and image benignity.
  arXiv  Detail & Related papers  (2023-03-27T19:23:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.