RT-Attack: Jailbreaking Text-to-Image Models via Random Token

• URL: http://arxiv.org/abs/2408.13896v2
• Date: Tue, 27 Aug 2024 15:13:01 GMT
• Title: RT-Attack: Jailbreaking Text-to-Image Models via Random Token
• Authors: Sensen Gao, Xiaojun Jia, Yihao Huang, Ranjie Duan, Jindong Gu, Yang Liu, Qing Guo,
• Abstract summary: We introduce a two-stage query-based black-box attack method utilizing random search. In the first stage, we establish a preliminary prompt by maximizing the semantic similarity between the adversarial and target harmful prompts. In the second stage, we use this initial prompt to refine our approach, creating a detailed adversarial prompt aimed at jailbreaking.
• Score: 24.61198605177661
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, Text-to-Image(T2I) models have achieved remarkable success in image generation and editing, yet these models still have many potential issues, particularly in generating inappropriate or Not-Safe-For-Work(NSFW) content. Strengthening attacks and uncovering such vulnerabilities can advance the development of reliable and practical T2I models. Most of the previous works treat T2I models as white-box systems, using gradient optimization to generate adversarial prompts. However, accessing the model's gradient is often impossible in real-world scenarios. Moreover, existing defense methods, those using gradient masking, are designed to prevent attackers from obtaining accurate gradient information. While some black-box jailbreak attacks have been explored, these typically rely on simply replacing sensitive words, leading to suboptimal attack performance. To address this issue, we introduce a two-stage query-based black-box attack method utilizing random search. In the first stage, we establish a preliminary prompt by maximizing the semantic similarity between the adversarial and target harmful prompts. In the second stage, we use this initial prompt to refine our approach, creating a detailed adversarial prompt aimed at jailbreaking and maximizing the similarity in image features between the images generated from this prompt and those produced by the target harmful prompt. Extensive experiments validate the effectiveness of our method in attacking the latest prompt checkers, post-hoc image checkers, securely trained T2I models, and online commercial models.

Related papers

• Multimodal Pragmatic Jailbreak on Text-to-image Models [43.67831238116829]
  This work introduces a novel type of jailbreak, which triggers T2I models to generate the image with visual text. We benchmark nine representative T2I models, including two close-source commercial models. All tested models suffer from such type of jailbreak, with rates of unsafe generation ranging from 8% to 74%.
  arXiv  Detail & Related papers  (2024-09-27T21:23:46Z)
• Perception-guided Jailbreak against Text-to-Image Models [18.825079959947857]
  We propose an LLM-driven perception-guided jailbreak method, termed PGJ. It is a black-box jailbreak method that requires no specific T2I model (model-free) and generates highly natural attack prompts. The experiments conducted on six open-source models and commercial online services with thousands of prompts have verified the effectiveness of PGJ.
  arXiv  Detail & Related papers  (2024-08-20T13:40:25Z)
• DiffZOO: A Purely Query-Based Black-Box Attack for Red-teaming Text-to-Image Generative Model via Zeroth Order Optimization [20.958826487430194]
  Red teaming attack methods are proposed to enhance or expose the T2I model's capability to generate unsuitable content. We propose DiffZOO which applies Zeroth Order Optimization to procure gradient approximations and harnesses both C-PRV and D-PRV to enhance attack prompts. Experiments on multiple state-of-the-art safety mechanisms show that DiffZOO attains an 8.5% higher average attack success rate than previous works.
  arXiv  Detail & Related papers  (2024-08-18T03:16:59Z)
• Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt [60.54666043358946]
  This paper introduces the Bi-Modal Adversarial Prompt Attack (BAP), which executes jailbreaks by optimizing textual and visual prompts cohesively. In particular, we utilize a large language model to analyze jailbreak failures and employ chain-of-thought reasoning to refine textual prompts.
  arXiv  Detail & Related papers  (2024-06-06T13:00:42Z)
• Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models [10.70975463369742]
  We present the Jailbreaking Prompt Attack (JPA) JPA searches for the target malicious concepts in the text embedding space using a group of antonyms. A prefix prompt is optimized in the discrete vocabulary space to align malicious concepts semantically in the text embedding space.
  arXiv  Detail & Related papers  (2024-04-02T09:49:35Z)
• Automated Black-box Prompt Engineering for Personalized Text-to-Image Generation [150.57983348059528]
  PRISM is an algorithm that automatically identifies human-interpretable and transferable prompts. It can effectively generate desired concepts given only black-box access to T2I models. Our experiments demonstrate the versatility and effectiveness of PRISM in generating accurate prompts for objects, styles and images.
  arXiv  Detail & Related papers  (2024-03-28T02:35:53Z)
• GuardT2I: Defending Text-to-Image Models from Adversarial Prompts [16.317849859000074]
  GuardT2I is a novel moderation framework that adopts a generative approach to enhance T2I models' robustness against adversarial prompts. Our experiments reveal that GuardT2I outperforms leading commercial solutions like OpenAI-Moderation and Microsoft Azure Moderator.
  arXiv  Detail & Related papers  (2024-03-03T09:04:34Z)
• Direct Consistency Optimization for Compositional Text-to-Image Personalization [73.94505688626651]
  Text-to-image (T2I) diffusion models, when fine-tuned on a few personal images, are able to generate visuals with a high degree of consistency. We propose to fine-tune the T2I model by maximizing consistency to reference images, while penalizing the deviation from the pretrained model.
  arXiv  Detail & Related papers  (2024-02-19T09:52:41Z)
• Cheating Suffix: Targeted Attack to Text-To-Image Diffusion Models with Multi-Modal Priors [59.43303903348258]
  Diffusion models have been widely deployed in various image generation tasks. They face challenges of being maliciously exploited to generate harmful or sensitive images. We propose a targeted attack method named MMP-Attack.
  arXiv  Detail & Related papers  (2024-02-02T12:39:49Z)
• If at First You Don't Succeed, Try, Try Again: Faithful Diffusion-based Text-to-Image Generation by Selection [53.320946030761796]
  diffusion-based text-to-image (T2I) models can lack faithfulness to the text prompt. We show that large T2I diffusion models are more faithful than usually assumed, and can generate images faithful to even complex prompts. We introduce a pipeline that generates candidate images for a text prompt and picks the best one according to an automatic scoring system.
  arXiv  Detail & Related papers  (2023-05-22T17:59:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.