Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs

• URL: http://arxiv.org/abs/2406.09324v3
• Date: Wed, 06 Nov 2024 04:43:57 GMT
• Title: Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs
• Authors: Zhao Xu, Fan Liu, Hao Liu,
• Abstract summary: Large Language Models (LLMs) have demonstrated significant capabilities in executing complex tasks in a zero-shot manner. LLMs are susceptible to jailbreak attacks and can be manipulated to produce harmful outputs.
• Score: 13.317364896194903
• License:
• Abstract: Although Large Language Models (LLMs) have demonstrated significant capabilities in executing complex tasks in a zero-shot manner, they are susceptible to jailbreak attacks and can be manipulated to produce harmful outputs. Recently, a growing body of research has categorized jailbreak attacks into token-level and prompt-level attacks. However, previous work primarily overlooks the diverse key factors of jailbreak attacks, with most studies concentrating on LLM vulnerabilities and lacking exploration of defense-enhanced LLMs. To address these issues, we introduced $\textbf{JailTrickBench}$ to evaluate the impact of various attack settings on LLM performance and provide a baseline for jailbreak attacks, encouraging the adoption of a standardized evaluation framework. Specifically, we evaluate the eight key factors of implementing jailbreak attacks on LLMs from both target-level and attack-level perspectives. We further conduct seven representative jailbreak attacks on six defense methods across two widely used datasets, encompassing approximately 354 experiments with about 55,000 GPU hours on A800-80G. Our experimental results highlight the need for standardized benchmarking to evaluate these attacks on defense-enhanced LLMs. Our code is available at https://github.com/usail-hkust/JailTrickBench.

Related papers

• SQL Injection Jailbreak: a structural disaster of large language models [71.55108680517422]
  We propose a novel jailbreak method, which utilizes the construction of input prompts by LLMs to inject jailbreak information into user prompts. Our SIJ method achieves nearly 100% attack success rates on five well-known open-source LLMs in the context of AdvBench.
  arXiv  Detail & Related papers  (2024-11-03T13:36:34Z)
• HSF: Defending against Jailbreak Attacks with Hidden State Filtering [14.031010511732008]
  We propose a jailbreak attack defense strategy based on a Hidden State Filter (HSF) HSF enables the model to preemptively identify and reject adversarial inputs before the inference process begins. It significantly reduces the success rate of jailbreak attacks while minimally impacting responses to benign user queries.
  arXiv  Detail & Related papers  (2024-08-31T06:50:07Z)
• EnJa: Ensemble Jailbreak on Large Language Models [69.13666224876408]
  Large Language Models (LLMs) are increasingly being deployed in safety-critical applications. LLMs can still be jailbroken by carefully crafted malicious prompts, producing content that violates policy regulations. We propose a novel EnJa attack to hide harmful instructions using prompt-level jailbreak, boost the attack success rate using a gradient-based attack, and connect the two types of jailbreak attacks via a template-based connector.
  arXiv  Detail & Related papers  (2024-08-07T07:46:08Z)
• Figure it Out: Analyzing-based Jailbreak Attack on Large Language Models [21.252514293436437]
  We propose Analyzing-based Jailbreak (ABJ) to combat jailbreak attacks on Large Language Models (LLMs) ABJ achieves 94.8% attack success rate (ASR) and 1.06 attack efficiency (AE) on GPT-4-turbo-0409, demonstrating state-of-the-art attack effectiveness and efficiency.
  arXiv  Detail & Related papers  (2024-07-23T06:14:41Z)
• JailBreakV-28K: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks [24.69275959735538]
  This paper investigates whether techniques that successfully jailbreak Large Language Models can be equally effective in jailbreaking MLLMs. We introduce JailBreakV-28K, a pioneering benchmark designed to assess the transferability of LLM jailbreak techniques to MLLMs. We generate 20, 000 text-based jailbreak prompts using advanced jailbreak attacks on LLMs, alongside 8, 000 image-based jailbreak inputs from recent MLLMs jailbreak attacks.
  arXiv  Detail & Related papers  (2024-04-03T19:23:18Z)
• EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models [53.87416566981008]
  This paper introduces EasyJailbreak, a unified framework simplifying the construction and evaluation of jailbreak attacks against Large Language Models (LLMs) It builds jailbreak attacks using four components: Selector, Mutator, Constraint, and Evaluator. Our validation across 10 distinct LLMs reveals a significant vulnerability, with an average breach probability of 60% under various jailbreaking attacks.
  arXiv  Detail & Related papers  (2024-03-18T18:39:53Z)
• Coercing LLMs to do and reveal (almost) anything [80.8601180293558]
  It has been shown that adversarial attacks on large language models (LLMs) can "jailbreak" the model into making harmful statements. We argue that the spectrum of adversarial attacks on LLMs is much larger than merely jailbreaking.
  arXiv  Detail & Related papers  (2024-02-21T18:59:13Z)
• Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues [16.97760778679782]
  We propose an indirect jailbreak attack approach, Puzzler, which can bypass the LLM's defense strategy and obtain malicious response. Our experiments show that Puzzler achieves a query success rate of 96.6% on closed-source LLMs. When tested against the state-of-the-art jailbreak detection approaches, Puzzler proves to be more effective at evading detection compared to baselines.
  arXiv  Detail & Related papers  (2024-02-14T11:11:51Z)
• Comprehensive Assessment of Jailbreak Attacks Against LLMs [28.58973312098698]
  We study 13 cutting-edge jailbreak methods from four categories, 160 questions from 16 violation categories, and six popular LLMs. Our experimental results demonstrate that the optimized jailbreak prompts consistently achieve the highest attack success rates. We discuss the trade-off between the attack performance and efficiency, as well as show that the transferability of the jailbreak prompts is still viable.
  arXiv  Detail & Related papers  (2024-02-08T13:42:50Z)
• Weak-to-Strong Jailbreaking on Large Language Models [96.50953637783581]
  Large language models (LLMs) are vulnerable to jailbreak attacks. Existing jailbreaking methods are computationally costly. We propose the weak-to-strong jailbreaking attack.
  arXiv  Detail & Related papers  (2024-01-30T18:48:37Z)
• Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization [98.18718484152595]
  We propose to integrate goal prioritization at both training and inference stages to counteract the intrinsic conflict between the goals of being helpful and ensuring safety. Our work thus contributes to the comprehension of jailbreaking attacks and defenses, and sheds light on the relationship between LLMs' capability and safety.
  arXiv  Detail & Related papers  (2023-11-15T16:42:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.