OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing

• URL: http://arxiv.org/abs/2605.07414v1
• Date: Fri, 08 May 2026 08:06:23 GMT
• Title: OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing
• Abstract summary: OrchJail is an orchestration-guided fuzzing framework for jailbreaking tool-calling T2I agents.<n>Our work highlights tool orchestration as a critical, previously unexplored attack surface.
• Score: 16.04943411061416
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Tool-calling text-to-image (T2I) agents can plan and execute multi-step tool chains to accomplish complex generation and editing queries. However, this capability introduces a new safety attack surface: harmful outputs may arise from tool orchestration, where individually benign steps combine into unsafe results, making prompt-only jailbreak techniques insufficient. We present OrchJail, an orchestration-guided fuzzing framework for jailbreaking tool-calling T2I agents. Its core idea is to exploit high-risk tool-orchestration patterns: by learning from successful jailbreak tool-calling traces and their causal relationships to prompt wording, OrchJail directly guides the fuzzing search toward prompts that are more likely to trigger unsafe multi-step tool behaviors, rather than relying on surface-level textual perturbations. Extensive experiments demonstrate that OrchJail improves jailbreak effectiveness and efficiency across representative toolcalling T2I agents, achieving higher attack success rates, better image fidelity, and lower query costs, while remaining robust against common jailbreak defenses. Our work highlights tool orchestration as a critical, previously unexplored attack surface and provides a novel framework for uncovering safety risks in T2I agents.

Related papers

• JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimization [6.473799439592539]
  JailbreakOPT is a tool-assisted framework for improving iterative single-turn jailbreak prompt optimization.<n>JailbreakOPT organizes diverse atomic jailbreak prompts into an attack tool library and composes them through a unified intra-episode optimization abstraction.<n> Experiments show that JailbreakOPT improves attack success rate (ASR) while reducing the number of attacks until success (No.A) compared with atomic single-turn attacks.
  arXiv  Detail & Related papers  (2026-06-09T20:22:29Z)
• Context-Fractured Decomposition Attacks on Tool-Using LLM Agents: Exploiting Artifact Provenance Gaps [10.07589034106101]
  jailbreak defenses must reason about cross-step composition rather than isolated text.<n>We operationalize a deployment failure mode for tool-using LLM agents, the emphprovenance gap<n>We instrument the failure mode with trace-level diagnostics and outline a verifiable mitigation direction.
  arXiv  Detail & Related papers  (2026-06-08T06:29:42Z)
• A Causal Perspective for Enhancing Jailbreak Attack and Defense [29.669194815878768]
  We propose a framework that integrates large language models into data-driven causal discovery.<n>We introduce a comprehensive dataset comprising 35k jailbreak attempts across seven language models.<n>Our analysis reveals that specific features, such as "Positive Character" and "Number of Task Steps", act as direct causal drivers of jailbreaks.
  arXiv  Detail & Related papers  (2026-01-31T15:20:13Z)
• The Trojan Example: Jailbreaking LLMs through Template Filling and Unsafety Reasoning [47.85771791033142]
  TrojFill is a black-box jailbreak that reframes unsafe instruction as a template-filling task.<n>We evaluate TrojFill on standard jailbreak benchmarks across leading Large Language Models.<n> generated prompts exhibit improved interpretability and transferability compared with prior black-box optimization approaches.
  arXiv  Detail & Related papers  (2025-10-24T06:43:10Z)
• Stand on The Shoulders of Giants: Building JailExpert from Previous Attack Experience [36.525169416008886]
  Large language models (LLMs) generate human-aligned content under certain safety constraints.<n>The textbfJailExpert framework is the first to achieve a formal representation of experience structure.<n>JailExpert achieves an average increase of 17% in attack success rate and 2.7 times improvement in attack efficiency.
  arXiv  Detail & Related papers  (2025-08-25T14:16:30Z)
• Anyone Can Jailbreak: Prompt-Based Attacks on LLMs and T2Is [8.214994509812724]
  Large language models (LLMs) and text-to-image (T2I) systems remain vulnerable to prompt-based attacks known as jailbreaks.<n>This paper presents a systems-style investigation into how non-experts reliably circumvent safety mechanisms.<n>We propose a unified taxonomy of prompt-level jailbreak strategies spanning both text-output and T2I models.
  arXiv  Detail & Related papers  (2025-07-29T13:55:23Z)
• TombRaider: Entering the Vault of History to Jailbreak Large Language Models [20.21399377784112]
  We introduce TombRaider, a novel jailbreak technique that exploits the ability to store, retrieve, and use historical knowledge of LLMs.<n>We intensively evaluated TombRaider on six popular models.<n> Experimental results showed that TombRaider could outperform state-of-the-art jailbreak techniques.
  arXiv  Detail & Related papers  (2025-01-27T14:12:07Z)
• AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents [84.96249955105777]
  LLM agents may pose a greater risk if misused, but their robustness remains underexplored.<n>We propose a new benchmark called AgentHarm to facilitate research on LLM agent misuse.<n>We find leading LLMs are surprisingly compliant with malicious agent requests without jailbreaking.
  arXiv  Detail & Related papers  (2024-10-11T17:39:22Z)
• EnJa: Ensemble Jailbreak on Large Language Models [69.13666224876408]
  Large Language Models (LLMs) are increasingly being deployed in safety-critical applications. LLMs can still be jailbroken by carefully crafted malicious prompts, producing content that violates policy regulations. We propose a novel EnJa attack to hide harmful instructions using prompt-level jailbreak, boost the attack success rate using a gradient-based attack, and connect the two types of jailbreak attacks via a template-based connector.
  arXiv  Detail & Related papers  (2024-08-07T07:46:08Z)
• Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models [15.582860145268553]
  JailFuzzer is a novel fuzzing framework driven by large language model (LLM) agents.<n>It generates natural and semantically coherent prompts, reducing the likelihood of detection by traditional defenses.<n>It achieves a high success rate in jailbreak attacks with minimal query overhead.
  arXiv  Detail & Related papers  (2024-08-01T12:54:46Z)
• WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models [66.34505141027624]
  We introduce WildTeaming, an automatic LLM safety red-teaming framework that mines in-the-wild user-chatbot interactions to discover 5.7K unique clusters of novel jailbreak tactics. WildTeaming reveals previously unidentified vulnerabilities of frontier LLMs, resulting in up to 4.6x more diverse and successful adversarial attacks.
  arXiv  Detail & Related papers  (2024-06-26T17:31:22Z)
• Automatic Jailbreaking of the Text-to-Image Generative AI Systems [76.9697122883554]
  We study the safety of the commercial T2I generation systems, such as ChatGPT, Copilot, and Gemini, on copyright infringement with naive prompts. We propose a stronger automated jailbreaking pipeline for T2I generation systems, which produces prompts that bypass their safety guards. Our framework successfully jailbreaks the ChatGPT with 11.0% block rate, making it generate copyrighted contents in 76% of the time.
  arXiv  Detail & Related papers  (2024-05-26T13:32:24Z)
• A StrongREJECT for Empty Jailbreaks [72.8807309802266]
  StrongREJECT is a high-quality benchmark for evaluating jailbreak performance. It scores the harmfulness of a victim model's responses to forbidden prompts. It achieves state-of-the-art agreement with human judgments of jailbreak effectiveness.
  arXiv  Detail & Related papers  (2024-02-15T18:58:09Z)
• AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models [54.95912006700379]
  We introduce AutoDAN, a novel jailbreak attack against aligned Large Language Models. AutoDAN can automatically generate stealthy jailbreak prompts by the carefully designed hierarchical genetic algorithm.
  arXiv  Detail & Related papers  (2023-10-03T19:44:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.