PLA: Prompt Learning Attack against Text-to-Image Generative Models

• URL: http://arxiv.org/abs/2508.03696v1
• Date: Mon, 14 Jul 2025 11:57:16 GMT
• Title: PLA: Prompt Learning Attack against Text-to-Image Generative Models
• Authors: Xinqi Lyu, Yihao Liu, Yanjie Li, Bin Xiao,
• Abstract summary: Text-to-Image (T2I) models pose significant risks of generating Not-Safe-For-Work (NSFW) content.<n>This paper proposes a novel prompt learning attack framework (PLA) to facilitate the learning of adversarial prompts in black-box settings.<n>Experiments show that our new method can effectively attack the safety mechanisms of black-box T2I models with a high success rate.
• Score: 11.86785431944315
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-Image (T2I) models have gained widespread adoption across various applications. Despite the success, the potential misuse of T2I models poses significant risks of generating Not-Safe-For-Work (NSFW) content. To investigate the vulnerability of T2I models, this paper delves into adversarial attacks to bypass the safety mechanisms under black-box settings. Most previous methods rely on word substitution to search adversarial prompts. Due to limited search space, this leads to suboptimal performance compared to gradient-based training. However, black-box settings present unique challenges to training gradient-driven attack methods, since there is no access to the internal architecture and parameters of T2I models. To facilitate the learning of adversarial prompts in black-box settings, we propose a novel prompt learning attack framework (PLA), where insightful gradient-based training tailored to black-box T2I models is designed by utilizing multimodal similarities. Experiments show that our new method can effectively attack the safety mechanisms of black-box T2I models including prompt filters and post-hoc safety checkers with a high success rate compared to state-of-the-art methods. Warning: This paper may contain offensive model-generated content.

Related papers

• RL-MTJail: Reinforcement Learning for Automated Black-Box Multi-Turn Jailbreaking of Large Language Models [60.201244463046784]
  Large language models are vulnerable to jailbreak attacks.<n>This paper studies black-box multi-turn jailbreaks, aiming to train attacker LLMs to elicit harmful content from black-box models.
  arXiv  Detail & Related papers  (2025-12-08T17:42:59Z)
• GenBreak: Red Teaming Text-to-Image Generators Using Large Language Models [65.91565607573786]
  Text-to-image (T2I) models can be misused to generate harmful content, including nudity or violence.<n>Recent research on red-teaming and adversarial attacks against T2I models has notable limitations.<n>We propose GenBreak, a framework that fine-tunes a red-team large language model (LLM) to systematically explore underlying vulnerabilities.
  arXiv  Detail & Related papers  (2025-06-11T09:09:12Z)
• T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attacks [67.91652526657599]
  We formalize the T2V jailbreak attack as a discrete optimization problem and propose a joint objective-based optimization framework, called T2V-OptJail.<n>We conduct large-scale experiments on several T2V models, covering both open-source models and real commercial closed-source models.<n>The proposed method improves 11.4% and 10.0% over the existing state-of-the-art method in terms of attack success rate.
  arXiv  Detail & Related papers  (2025-05-10T16:04:52Z)
• In-Context Experience Replay Facilitates Safety Red-Teaming of Text-to-Image Diffusion Models [104.94706600050557]
  Text-to-image (T2I) models have shown remarkable progress, but their potential to generate harmful content remains a critical concern in the ML community.<n>We propose ICER, a novel red-teaming framework that generates interpretable and semantic meaningful problematic prompts.<n>Our work provides crucial insights for developing more robust safety mechanisms in T2I systems.
  arXiv  Detail & Related papers  (2024-11-25T04:17:24Z)
• HTS-Attack: Heuristic Token Search for Jailbreaking Text-to-Image Models [28.28898114141277]
  Text-to-Image(T2I) models have achieved remarkable success in image generation and editing.<n>These models still have many potential issues, particularly in generating inappropriate or Not-Safe-For-Work(NSFW) content.<n>We propose HTS-Attack, a token search attack method.
  arXiv  Detail & Related papers  (2024-08-25T17:33:40Z)
• DiffZOO: A Purely Query-Based Black-Box Attack for Red-teaming Text-to-Image Generative Model via Zeroth Order Optimization [20.958826487430194]
  Red teaming attack methods are proposed to enhance or expose the T2I model's capability to generate unsuitable content.<n>We propose DiffZOO which applies Zeroth Order Optimization to procure gradient approximations and harnesses both C-PRV and D-PRV to enhance attack prompts.<n>Experiments on multiple state-of-the-art safety mechanisms show that DiffZOO attains an 8.5% higher average attack success rate than previous works.
  arXiv  Detail & Related papers  (2024-08-18T03:16:59Z)
• Can Reinforcement Learning Unlock the Hidden Dangers in Aligned Large Language Models? [3.258629327038072]
  Large Language Models (LLMs) have demonstrated impressive capabilities in natural language tasks. Yet, the potential for generating harmful content through these models seems to persist. This paper explores the concept of jailbreaking LLMs-reversing their alignment through adversarial triggers.
  arXiv  Detail & Related papers  (2024-08-05T17:27:29Z)
• Automated Black-box Prompt Engineering for Personalized Text-to-Image Generation [149.96612254604986]
  PRISM is an algorithm that automatically produces human-interpretable and transferable prompts.<n>Inspired by large language model (LLM) jailbreaking, PRISM leverages the in-context learning ability of LLMs to iteratively refine the candidate prompt distribution.<n>Our experiments demonstrate the versatility and effectiveness of PRISM in generating accurate prompts for objects, styles, and images across multiple T2I models.
  arXiv  Detail & Related papers  (2024-03-28T02:35:53Z)
• GuardT2I: Defending Text-to-Image Models from Adversarial Prompts [16.317849859000074]
  GuardT2I is a novel moderation framework that adopts a generative approach to enhance T2I models' robustness against adversarial prompts. Our experiments reveal that GuardT2I outperforms leading commercial solutions like OpenAI-Moderation and Microsoft Azure Moderator.
  arXiv  Detail & Related papers  (2024-03-03T09:04:34Z)
• Cross-Modal Transferable Adversarial Attacks from Images to Videos [82.0745476838865]
  Recent studies have shown that adversarial examples hand-crafted on one white-box model can be used to attack other black-box models. We propose a simple yet effective cross-modal attack method, named as Image To Video (I2V) attack. I2V generates adversarial frames by minimizing the cosine similarity between features of pre-trained image models from adversarial and benign examples.
  arXiv  Detail & Related papers  (2021-12-10T08:19:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.