HTS-Attack: Heuristic Token Search for Jailbreaking Text-to-Image Models

• URL: http://arxiv.org/abs/2408.13896v3
• Date: Sun, 15 Dec 2024 05:13:26 GMT
• Title: HTS-Attack: Heuristic Token Search for Jailbreaking Text-to-Image Models
• Authors: Sensen Gao, Xiaojun Jia, Yihao Huang, Ranjie Duan, Jindong Gu, Yang Bai, Yang Liu, Qing Guo,
• Abstract summary: Text-to-Image(T2I) models have achieved remarkable success in image generation and editing. These models still have many potential issues, particularly in generating inappropriate or Not-Safe-For-Work(NSFW) content. We propose HTS-Attack, a token search attack method.
• Score: 28.28898114141277
• License:
• Abstract: Text-to-Image(T2I) models have achieved remarkable success in image generation and editing, yet these models still have many potential issues, particularly in generating inappropriate or Not-Safe-For-Work(NSFW) content. Strengthening attacks and uncovering such vulnerabilities can advance the development of reliable and practical T2I models. Most of the previous works treat T2I models as white-box systems, using gradient optimization to generate adversarial prompts. However, accessing the model's gradient is often impossible in real-world scenarios. Moreover, existing defense methods, those using gradient masking, are designed to prevent attackers from obtaining accurate gradient information. While several black-box jailbreak attacks have been explored, they achieve the limited performance of jailbreaking T2I models due to difficulties associated with optimization in discrete spaces. To address this, we propose HTS-Attack, a heuristic token search attack method. HTS-Attack begins with an initialization that removes sensitive tokens, followed by a heuristic search where high-performing candidates are recombined and mutated. This process generates a new pool of candidates, and the optimal adversarial prompt is updated based on their effectiveness. By incorporating both optimal and suboptimal candidates, HTS-Attack avoids local optima and improves robustness in bypassing defenses. Extensive experiments validate the effectiveness of our method in attacking the latest prompt checkers, post-hoc image checkers, securely trained T2I models, and online commercial models.

Related papers

• In-Context Experience Replay Facilitates Safety Red-Teaming of Text-to-Image Diffusion Models [104.94706600050557]
  Text-to-image (T2I) models have shown remarkable progress, but their potential to generate harmful content remains a critical concern in the ML community. We propose ICER, a novel red-teaming framework that generates interpretable and semantic meaningful problematic prompts. Our work provides crucial insights for developing more robust safety mechanisms in T2I systems.
  arXiv  Detail & Related papers  (2024-11-25T04:17:24Z)
• A Realistic Threat Model for Large Language Model Jailbreaks [87.64278063236847]
  In this work, we propose a unified threat model for the principled comparison of jailbreak attacks. Our threat model combines constraints in perplexity, measuring how far a jailbreak deviates from natural text. We adapt popular attacks to this new, realistic threat model, with which we, for the first time, benchmark these attacks on equal footing.
  arXiv  Detail & Related papers  (2024-10-21T17:27:01Z)
• Multimodal Pragmatic Jailbreak on Text-to-image Models [43.67831238116829]
  This work introduces a novel type of jailbreak, which triggers T2I models to generate the image with visual text. We benchmark nine representative T2I models, including two close-source commercial models. All tested models suffer from such type of jailbreak, with rates of unsafe generation ranging from 8% to 74%.
  arXiv  Detail & Related papers  (2024-09-27T21:23:46Z)
• DiffZOO: A Purely Query-Based Black-Box Attack for Red-teaming Text-to-Image Generative Model via Zeroth Order Optimization [20.958826487430194]
  Red teaming attack methods are proposed to enhance or expose the T2I model's capability to generate unsuitable content. We propose DiffZOO which applies Zeroth Order Optimization to procure gradient approximations and harnesses both C-PRV and D-PRV to enhance attack prompts. Experiments on multiple state-of-the-art safety mechanisms show that DiffZOO attains an 8.5% higher average attack success rate than previous works.
  arXiv  Detail & Related papers  (2024-08-18T03:16:59Z)
• UPAM: Unified Prompt Attack in Text-to-Image Generation Models Against Both Textual Filters and Visual Checkers [21.30197653947112]
  Text-to-Image (T2I) models have raised security concerns due to their potential to generate inappropriate or harmful images. We propose UPAM, a novel framework that investigates the robustness of T2I models from the attack perspective.
  arXiv  Detail & Related papers  (2024-05-18T16:47:36Z)
• Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models [10.70975463369742]
  We present the Jailbreaking Prompt Attack (JPA) JPA searches for the target malicious concepts in the text embedding space using a group of antonyms. A prefix prompt is optimized in the discrete vocabulary space to align malicious concepts semantically in the text embedding space.
  arXiv  Detail & Related papers  (2024-04-02T09:49:35Z)
• Automated Black-box Prompt Engineering for Personalized Text-to-Image Generation [149.96612254604986]
  PRISM is an algorithm that automatically identifies human-interpretable and transferable prompts. It can effectively generate desired concepts given only black-box access to T2I models. Our experiments demonstrate the versatility and effectiveness of PRISM in generating accurate prompts for objects, styles and images.
  arXiv  Detail & Related papers  (2024-03-28T02:35:53Z)
• GuardT2I: Defending Text-to-Image Models from Adversarial Prompts [16.317849859000074]
  GuardT2I is a novel moderation framework that adopts a generative approach to enhance T2I models' robustness against adversarial prompts. Our experiments reveal that GuardT2I outperforms leading commercial solutions like OpenAI-Moderation and Microsoft Azure Moderator.
  arXiv  Detail & Related papers  (2024-03-03T09:04:34Z)
• Orthogonal Deep Models As Defense Against Black-Box Attacks [71.23669614195195]
  We study the inherent weakness of deep models in black-box settings where the attacker may develop the attack using a model similar to the targeted model. We introduce a novel gradient regularization scheme that encourages the internal representation of a deep model to be orthogonal to another. We verify the effectiveness of our technique on a variety of large-scale models.
  arXiv  Detail & Related papers  (2020-06-26T08:29:05Z)
• BERT-ATTACK: Adversarial Attack Against BERT Using BERT [77.82947768158132]
  Adrial attacks for discrete data (such as texts) are more challenging than continuous data (such as images) We propose textbfBERT-Attack, a high-quality and effective method to generate adversarial samples. Our method outperforms state-of-the-art attack strategies in both success rate and perturb percentage.
  arXiv  Detail & Related papers  (2020-04-21T13:30:02Z)
• Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent [92.4348499398224]
  Black-box adversarial attack methods have received special attentions owing to their practicality and simplicity. We propose a zeroth-order natural gradient descent (ZO-NGD) method to design the adversarial attacks. ZO-NGD can obtain significantly lower model query complexities compared with state-of-the-art attack methods.
  arXiv  Detail & Related papers  (2020-02-18T21:48:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.