A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination

• URL: http://arxiv.org/abs/2608.04034v1
• Date: Mon, 03 Aug 2026 02:28:52 GMT
• Title: A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination
• Abstract summary: Multimodal Large Language Models (MLLMs) have achieved impressive progress in image-text comprehension and generation.<n>They remain susceptible to jailbreak attacks that can trigger harmful outputs and pose serious safety concerns.<n>Existing multimodal jailbreak attacks have shown the feasibility of such attacks, but they still face two fundamental challenges.
• Score: 24.819747080813485
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal Large Language Models (MLLMs) have achieved impressive progress in image-text comprehension and generation, yet they remain susceptible to jailbreak attacks that can trigger harmful outputs and pose serious safety concerns. Existing multimodal jailbreak attacks have shown the feasibility of such attacks, but they still face two fundamental challenges: the lack of a atomic multi-modal strategy space, the absence of a concise and efficient executable framework beyond human-craft experience. To address these challenges, we first decompose the text-image jailbreak strategy space into three levels: structural, semantic, and syntactic, constructing a jailbreak strategy set encompassing both text and image modalities to systematically achieve combined coverage of different attack types. Then we propose a multimodal automated red team jailbreak method named Hierarchical Atomic Combination Attack (HACA). Specifically, based on a six-dimensional strategy space, a cross-modal joint planner is used to select and combine the different atomic jailbreak strategy for subsequent jailbreak command generation. Finally, at the implementation level, we explore to apply a unified generate executor to directly generate jailbreak instructions based on the selected multi-modal strategies. A series of experiments show that our automated red team method can achieve an attack success rate of average 95.48\% against five mainstream MLLMs.

Related papers

• Guarding the Guardrails: A Taxonomy-Driven Approach to Jailbreak Detection [1.8374319565577155]
  Jailbreaking techniques pose a significant threat to the safety of Large Language Models.<n>To advance the understanding of the effectiveness of jailbreaking techniques, we conducted a structured red-teaming challenge.<n>We developed a comprehensive hierarchical taxonomy of 50 jailbreak strategies, consolidating and extending prior classifications into seven broad families.
  arXiv  Detail & Related papers  (2025-10-14T12:34:41Z)
• Test-Time Immunization: A Universal Defense Framework Against Jailbreaks for (Multimodal) Large Language Models [80.66766532477973]
  Test-time IMmunization (TIM) can adaptively defend against various jailbreak attacks in a self-evolving way.<n>Test-time IMmunization (TIM) can adaptively defend against various jailbreak attacks in a self-evolving way.
  arXiv  Detail & Related papers  (2025-05-28T11:57:46Z)
• Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models [1.0768089718297513]
  This paper proposes a multimodal jailbreaking method: JMLLM.<n>It integrates multiple strategies to perform comprehensive jailbreak attacks across text, visual, and auditory modalities.<n>We also contribute a new and comprehensive dataset for multimodal jailbreaking research: TriJail.
  arXiv  Detail & Related papers  (2024-12-21T09:43:51Z)
• Rapid Response: Mitigating LLM Jailbreaks with a Few Examples [13.841146655178585]
  We develop rapid response techniques to look to block whole classes of jailbreaks after observing only a handful of attacks. We evaluate five rapid response methods, all of which use jailbreak proliferation. Our strongest method reduces attack success rate by a factor greater than 240 on an in-distribution set of jailbreaks and a factor greater than 15 on an out-of-distribution set.
  arXiv  Detail & Related papers  (2024-11-12T02:44:49Z)
• IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves [70.43466586161345]
  IDEATOR is a novel jailbreak method that autonomously generates malicious image-text pairs for black-box jailbreak attacks.<n>Our benchmark results on 11 recently releasedVLMs reveal significant gaps in safety alignment.<n>For instance, our challenge set ASRs of 46.31% on GPT-4o and 19.65% on Claude-3.5-Sonnet, underscoring the urgent need for stronger defenses.
  arXiv  Detail & Related papers  (2024-10-29T07:15:56Z)
• PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMs [33.87649859430635]
  Large Language Models (LLMs) have excelled in various tasks but are still vulnerable to jailbreaking attacks.<n>PAPILLON is an automated, black-box jailbreaking attack framework.<n>It adapts the black-box fuzz testing approach with a series of customized designs.
  arXiv  Detail & Related papers  (2024-09-23T10:03:09Z)
• h4rm3l: A language for Composable Jailbreak Attack Synthesis [48.5611060845958]
  h4rm3l is a novel approach that addresses the gap with a human-readable domain-specific language.<n>We show that h4rm3l's synthesized attacks are diverse and more successful than existing jailbreak attacks in literature.
  arXiv  Detail & Related papers  (2024-08-09T01:45:39Z)
• EnJa: Ensemble Jailbreak on Large Language Models [69.13666224876408]
  Large Language Models (LLMs) are increasingly being deployed in safety-critical applications. LLMs can still be jailbroken by carefully crafted malicious prompts, producing content that violates policy regulations. We propose a novel EnJa attack to hide harmful instructions using prompt-level jailbreak, boost the attack success rate using a gradient-based attack, and connect the two types of jailbreak attacks via a template-based connector.
  arXiv  Detail & Related papers  (2024-08-07T07:46:08Z)
• Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models [15.582860145268553]
  JailFuzzer is a novel fuzzing framework driven by large language model (LLM) agents.<n>It generates natural and semantically coherent prompts, reducing the likelihood of detection by traditional defenses.<n>It achieves a high success rate in jailbreak attacks with minimal query overhead.
  arXiv  Detail & Related papers  (2024-08-01T12:54:46Z)
• AutoJailbreak: Exploring Jailbreak Attacks and Defenses through a Dependency Lens [83.08119913279488]
  We present a systematic analysis of the dependency relationships in jailbreak attack and defense techniques. We propose three comprehensive, automated, and logical frameworks. We show that the proposed ensemble jailbreak attack and defense framework significantly outperforms existing research.
  arXiv  Detail & Related papers  (2024-06-06T07:24:41Z)
• EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models [53.87416566981008]
  This paper introduces EasyJailbreak, a unified framework simplifying the construction and evaluation of jailbreak attacks against Large Language Models (LLMs) It builds jailbreak attacks using four components: Selector, Mutator, Constraint, and Evaluator. Our validation across 10 distinct LLMs reveals a significant vulnerability, with an average breach probability of 60% under various jailbreaking attacks.
  arXiv  Detail & Related papers  (2024-03-18T18:39:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.