Graph of Attacks with Pruning: Optimizing Stealthy Jailbreak Prompt Generation for Enhanced LLM Content Moderation

• URL: http://arxiv.org/abs/2501.18638v2
• Date: Fri, 13 Jun 2025 15:44:43 GMT
• Title: Graph of Attacks with Pruning: Optimizing Stealthy Jailbreak Prompt Generation for Enhanced LLM Content Moderation
• Authors: Daniel Schwartz, Dmitriy Bespalov, Zhe Wang, Ninad Kulkarni, Yanjun Qi,
• Abstract summary: This paper introduces the GAP (Graph of Attacks with Pruning) framework for generating stealthy jailbreak prompts.<n>GAP addresses limitations in existing tree-based LLM jailbreak methods by implementing an interconnected graph structure.<n>We present specialized variants like GAP-Auto for automated seed generation and GAP-VLM for multimodal attacks.
• Score: 12.303965802578087
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As large language models (LLMs) become increasingly prevalent, ensuring their robustness against adversarial misuse is crucial. This paper introduces the GAP (Graph of Attacks with Pruning) framework, an advanced approach for generating stealthy jailbreak prompts to evaluate and enhance LLM safeguards. GAP addresses limitations in existing tree-based LLM jailbreak methods by implementing an interconnected graph structure that enables knowledge sharing across attack paths. Our experimental evaluation demonstrates GAP's superiority over existing techniques, achieving a 20.8% increase in attack success rates while reducing query costs by 62.7%. GAP consistently outperforms state-of-the-art methods for attacking both open and closed LLMs, with attack success rates of >96%. Additionally, we present specialized variants like GAP-Auto for automated seed generation and GAP-VLM for multimodal attacks. GAP-generated prompts prove highly effective in improving content moderation systems, increasing true positive detection rates by 108.5% and accuracy by 183.6% when used for fine-tuning. Our implementation is available at https://github.com/dsbuddy/GAP-LLM-Safety.

Related papers

• Advancing Jailbreak Strategies: A Hybrid Approach to Exploiting LLM Vulnerabilities and Bypassing Modern Defenses [4.706534644850809]
  Two primary inference-phase threats are token-level and prompt-level jailbreaks.<n>We propose two hybrid approaches that integrate token- and prompt-level techniques to enhance jailbreak effectiveness across diverse PTLMs.
  arXiv  Detail & Related papers  (2025-06-27T07:26:33Z)
• Prefill-Based Jailbreak: A Novel Approach of Bypassing LLM Safety Boundary [2.4329261266984346]
  Large Language Models (LLMs) are designed to generate helpful and safe content. adversarial attacks, commonly referred to as jailbreak, can bypass their safety protocols. We introduce a novel jailbreak attack method that leverages the prefilling feature of LLMs.
  arXiv  Detail & Related papers  (2025-04-28T07:38:43Z)
• Geneshift: Impact of different scenario shift on Jailbreaking LLM [55.26229741296822]
  We propose a black-box jailbreak attack termed GeneShift, by using a genetic algorithm to optimize the scenario shifts.<n>We show that GeneShift increases the jailbreak success rate from 0% to 60% when direct prompting alone would fail.
  arXiv  Detail & Related papers  (2025-04-10T20:02:35Z)
• Iterative Prompting with Persuasion Skills in Jailbreaking Large Language Models [2.1511703382556657]
  This study exploits large language models (LLMs) with an iterative prompting technique.<n>We analyze the response patterns of LLMs, including GPT-3.5, GPT-4, LLaMa2, Vicuna, and ChatGLM.<n>Persuading strategies enhance prompt effectiveness while maintaining consistency with malicious intent.
  arXiv  Detail & Related papers  (2025-03-26T08:40:46Z)
• Making Every Step Effective: Jailbreaking Large Vision-Language Models Through Hierarchical KV Equalization [74.78433600288776]
  HKVE (Hierarchical Key-Value Equalization) is an innovative jailbreaking framework that selectively accepts gradient optimization results.<n>We show that HKVE substantially outperforms existing methods by substantially outperforming existing methods by margins of 20.43%, 21.01% and 26.43% respectively.
  arXiv  Detail & Related papers  (2025-03-14T17:57:42Z)
• Exploiting the Index Gradients for Optimization-Based Jailbreaking on Large Language Models [16.83476701024932]
  Greedy Coordinate Gradient (GCG) method has demonstrated ability to automatically generate adversarial suffixes that jailbreak state-of-the-art LLMs.<n>We propose the Model Attack Gradient Index GCG (MAGIC) that addresses the Indirect Effect by exploiting the gradient information of the suffix tokens.<n>Experiments on AdvBench show that MAGIC achieves up to a 1.5x speedup, while maintaining Attack Success Rates (ASR) on par or even higher.
  arXiv  Detail & Related papers  (2024-12-11T18:37:56Z)
• IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves [64.46372846359694]
  IDEATOR is a novel jailbreak method that autonomously generates malicious image-text pairs for black-box jailbreak attacks. Our benchmark results on 11 recently releasedVLMs reveal significant gaps in safety alignment. For instance, our challenge set achieves ASRs of 46.31% on GPT-4o and 19.65% on Claude-3.5-Sonnet.
  arXiv  Detail & Related papers  (2024-10-29T07:15:56Z)
• ADVLLM: Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities [63.603861880022954]
  We introduce ADV-LLM, an iterative self-tuning process that crafts adversarial LLMs with enhanced jailbreak ability.<n>Our framework significantly reduces the computational cost of generating adversarial suffixes while achieving nearly 100% ASR on various open-source LLMs.<n>It exhibits strong attack transferability to closed-source models, achieving 99% ASR on GPT-3.5 and 49% ASR on GPT-4, despite being optimized solely on Llama3.
  arXiv  Detail & Related papers  (2024-10-24T06:36:12Z)
• Deciphering the Chaos: Enhancing Jailbreak Attacks via Adversarial Prompt Translation [71.92055093709924]
  We propose a novel method that "translates" garbled adversarial prompts into coherent and human-readable natural language adversarial prompts.<n>It also offers a new approach to discovering effective designs for jailbreak prompts, advancing the understanding of jailbreak attacks.<n>Our method achieves over 90% attack success rates against Llama-2-Chat models on AdvBench, despite their outstanding resistance to jailbreak attacks.
  arXiv  Detail & Related papers  (2024-10-15T06:31:04Z)
• PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMs [33.87649859430635]
  Large Language Models (LLMs) have excelled in various tasks but are still vulnerable to jailbreaking attacks.<n>In this paper, we introduce a novel jailbreaking attack framework called PAPILLON.<n>It is an automated, black-box jailbreaking attack framework that adapts the black-box fuzz testing approach with a series of customized designs.
  arXiv  Detail & Related papers  (2024-09-23T10:03:09Z)
• EnJa: Ensemble Jailbreak on Large Language Models [69.13666224876408]
  Large Language Models (LLMs) are increasingly being deployed in safety-critical applications. LLMs can still be jailbroken by carefully crafted malicious prompts, producing content that violates policy regulations. We propose a novel EnJa attack to hide harmful instructions using prompt-level jailbreak, boost the attack success rate using a gradient-based attack, and connect the two types of jailbreak attacks via a template-based connector.
  arXiv  Detail & Related papers  (2024-08-07T07:46:08Z)
• Improved Techniques for Optimization-Based Jailbreaking on Large Language Models [78.32176751215073]
  Greedy Coordinate Gradient (GCG) attack's success has led to a growing interest in the study of optimization-based jailbreaking techniques. We present several improved (empirical) techniques for optimization-based jailbreaks like GCG. The results demonstrate that our improved techniques can help GCG outperform state-of-the-art jailbreaking attacks and achieve nearly 100% attack success rate.
  arXiv  Detail & Related papers  (2024-05-31T17:07:15Z)
• AutoBreach: Universal and Adaptive Jailbreaking with Efficient Wordplay-Guided Optimization [31.034290076970205]
  We introduce AutoBreach, a novel method for jailbreaking large language models (LLMs) Inspired by the versatility of wordplay, AutoBreach employs a wordplay-guided mapping rule sampling strategy to generate adversarial prompts. AutoBreach can efficiently identify security vulnerabilities across various LLMs, including three proprietary models: Claude-3, GPT-3.5, GPT-4 Turbo, and two LLMs' web platforms: Bingchat, GPT-4 Web, achieving an average success rate of over 80% with fewer than 10 queries.
  arXiv  Detail & Related papers  (2024-05-30T03:38:31Z)
• Improved Generation of Adversarial Examples Against Safety-aligned LLMs [72.38072942860309]
  Adversarial prompts generated using gradient-based methods exhibit outstanding performance in performing automatic jailbreak attacks against safety-aligned LLMs. In this paper, we explore a new perspective on this problem, suggesting that it can be alleviated by leveraging innovations inspired in transfer-based attacks. We show that 87% of the query-specific adversarial suffixes generated by the developed combination can induce Llama-2-7B-Chat to produce the output that exactly matches the target string on AdvBench.
  arXiv  Detail & Related papers  (2024-05-28T06:10:12Z)
• Automatic Jailbreaking of the Text-to-Image Generative AI Systems [76.9697122883554]
  We study the safety of the commercial T2I generation systems, such as ChatGPT, Copilot, and Gemini, on copyright infringement with naive prompts. We propose a stronger automated jailbreaking pipeline for T2I generation systems, which produces prompts that bypass their safety guards. Our framework successfully jailbreaks the ChatGPT with 11.0% block rate, making it generate copyrighted contents in 76% of the time.
  arXiv  Detail & Related papers  (2024-05-26T13:32:24Z)
• PAL: Proxy-Guided Black-Box Attack on Large Language Models [55.57987172146731]
  Large Language Models (LLMs) have surged in popularity in recent months, but they have demonstrated capabilities to generate harmful content when manipulated. We introduce the Proxy-Guided Attack on LLMs (PAL), the first optimization-based attack on LLMs in a black-box query-only setting. Our attack achieves 84% attack success rate (ASR) on GPT-3.5-Turbo and 48% on Llama-2-7B, compared to 4% for the current state of the art.
  arXiv  Detail & Related papers  (2024-02-15T02:54:49Z)
• Pandora: Jailbreak GPTs by Retrieval Augmented Generation Poisoning [19.45092401994873]
  This study investigates indirect jailbreak attacks on Large Language Models(LLMs) We introduce a novel attack vector named Retrieval Augmented Generation Poisoning. Pandora exploits the synergy between LLMs and RAG through prompt manipulation to generate unexpected responses.
  arXiv  Detail & Related papers  (2024-02-13T12:40:39Z)
• Tree of Attacks: Jailbreaking Black-Box LLMs Automatically [34.36053833900958]
  We present Tree of Attacks with Pruning (TAP), an automated method for generating jailbreaks. TAP generates prompts that jailbreak state-of-the-art LLMs for more than 80% of the prompts. TAP is also capable of jailbreaking LLMs protected by state-of-the-art guardrails, e.g., LlamaGuard.
  arXiv  Detail & Related papers  (2023-12-04T18:49:23Z)
• Jailbreaking GPT-4V via Self-Adversarial Attacks with System Prompts [64.60375604495883]
  We discover a system prompt leakage vulnerability in GPT-4V. By employing GPT-4 as a red teaming tool against itself, we aim to search for potential jailbreak prompts leveraging stolen system prompts. We also evaluate the effect of modifying system prompts to defend against jailbreaking attacks.
  arXiv  Detail & Related papers  (2023-11-15T17:17:39Z)
• "Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models [50.22128133926407]
  We conduct a comprehensive analysis of 1,405 jailbreak prompts spanning from December 2022 to December 2023. We identify 131 jailbreak communities and discover unique characteristics of jailbreak prompts and their major attack strategies. We identify five highly effective jailbreak prompts that achieve 0.95 attack success rates on ChatGPT (GPT-3.5) and GPT-4.
  arXiv  Detail & Related papers  (2023-08-07T16:55:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.