Comprehensive Assessment of Jailbreak Attacks Against LLMs

• URL: http://arxiv.org/abs/2402.05668v1
• Date: Thu, 8 Feb 2024 13:42:50 GMT
• Title: Comprehensive Assessment of Jailbreak Attacks Against LLMs
• Authors: Junjie Chu and Yugeng Liu and Ziqing Yang and Xinyue Shen and Michael Backes and Yang Zhang
• Abstract summary: We study 13 cutting-edge jailbreak methods from four categories, 160 questions from 16 violation categories, and six popular LLMs. Our experimental results demonstrate that the optimized jailbreak prompts consistently achieve the highest attack success rates. We discuss the trade-off between the attack performance and efficiency, as well as show that the transferability of the jailbreak prompts is still viable.
• Score: 28.58973312098698
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Misuse of the Large Language Models (LLMs) has raised widespread concern. To address this issue, safeguards have been taken to ensure that LLMs align with social ethics. However, recent findings have revealed an unsettling vulnerability bypassing the safeguards of LLMs, known as jailbreak attacks. By applying techniques, such as employing role-playing scenarios, adversarial examples, or subtle subversion of safety objectives as a prompt, LLMs can produce an inappropriate or even harmful response. While researchers have studied several categories of jailbreak attacks, they have done so in isolation. To fill this gap, we present the first large-scale measurement of various jailbreak attack methods. We concentrate on 13 cutting-edge jailbreak methods from four categories, 160 questions from 16 violation categories, and six popular LLMs. Our extensive experimental results demonstrate that the optimized jailbreak prompts consistently achieve the highest attack success rates, as well as exhibit robustness across different LLMs. Some jailbreak prompt datasets, available from the Internet, can also achieve high attack success rates on many LLMs, such as ChatGLM3, GPT-3.5, and PaLM2. Despite the claims from many organizations regarding the coverage of violation categories in their policies, the attack success rates from these categories remain high, indicating the challenges of effectively aligning LLM policies and the ability to counter jailbreak attacks. We also discuss the trade-off between the attack performance and efficiency, as well as show that the transferability of the jailbreak prompts is still viable, becoming an option for black-box models. Overall, our research highlights the necessity of evaluating different jailbreak methods. We hope our study can provide insights for future research on jailbreak attacks and serve as a benchmark tool for evaluating them for practitioners.

Related papers

• Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense [55.77152277982117]
  We introduce Layer-AdvPatcher, a methodology designed to defend against jailbreak attacks. We use an unlearning strategy to patch specific layers within large language models through self-augmented datasets. Our framework reduces the harmfulness and attack success rate of jailbreak attacks.
  arXiv  Detail & Related papers  (2025-01-05T19:06:03Z)
• Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models [59.25318174362368]
  Jailbreaking in Large Language Models (LLMs) is a major security concern as it can deceive LLMs to generate harmful text. We conduct a detailed analysis of seven different jailbreak methods and find that disagreements stem from insufficient observation samples. We propose a novel defense called textbfActivation Boundary Defense (ABD), which adaptively constrains the activations within the safety boundary.
  arXiv  Detail & Related papers  (2024-12-22T14:18:39Z)
• Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment [97.38766396447369]
  Despite training-time safety alignment, MLLMs remain vulnerable to jailbreak attacks. We propose Immune, an inference-time defense framework that leverages a safe reward model to defend against jailbreak attacks.
  arXiv  Detail & Related papers  (2024-11-27T19:00:10Z)
• Rapid Response: Mitigating LLM Jailbreaks with a Few Examples [13.841146655178585]
  We develop rapid response techniques to look to block whole classes of jailbreaks after observing only a handful of attacks. We evaluate five rapid response methods, all of which use jailbreak proliferation. Our strongest method reduces attack success rate by a factor greater than 240 on an in-distribution set of jailbreaks and a factor greater than 15 on an out-of-distribution set.
  arXiv  Detail & Related papers  (2024-11-12T02:44:49Z)
• What Features in Prompts Jailbreak LLMs? Investigating the Mechanisms Behind Attacks [3.0700566896646047]
  We show that different jailbreaking methods work via different nonlinear features in prompts. These mechanistic jailbreaks are able to jailbreak Gemma-7B-IT more reliably than 34 of the 35 techniques that it was trained on.
  arXiv  Detail & Related papers  (2024-11-02T17:29:47Z)
• EnJa: Ensemble Jailbreak on Large Language Models [69.13666224876408]
  Large Language Models (LLMs) are increasingly being deployed in safety-critical applications. LLMs can still be jailbroken by carefully crafted malicious prompts, producing content that violates policy regulations. We propose a novel EnJa attack to hide harmful instructions using prompt-level jailbreak, boost the attack success rate using a gradient-based attack, and connect the two types of jailbreak attacks via a template-based connector.
  arXiv  Detail & Related papers  (2024-08-07T07:46:08Z)
• Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs [13.317364896194903]
  Large Language Models (LLMs) have demonstrated significant capabilities in executing complex tasks in a zero-shot manner. LLMs are susceptible to jailbreak attacks and can be manipulated to produce harmful outputs.
  arXiv  Detail & Related papers  (2024-06-13T17:01:40Z)
• JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Models [21.854909839996612]
  Jailbreak attacks induce Large Language Models (LLMs) to generate harmful responses. There is no consensus on evaluating jailbreaks. JailbreakEval is a toolkit for evaluating jailbreak attempts.
  arXiv  Detail & Related papers  (2024-06-13T16:59:43Z)
• EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models [53.87416566981008]
  This paper introduces EasyJailbreak, a unified framework simplifying the construction and evaluation of jailbreak attacks against Large Language Models (LLMs) It builds jailbreak attacks using four components: Selector, Mutator, Constraint, and Evaluator. Our validation across 10 distinct LLMs reveals a significant vulnerability, with an average breach probability of 60% under various jailbreaking attacks.
  arXiv  Detail & Related papers  (2024-03-18T18:39:53Z)
• "Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models [50.22128133926407]
  We conduct a comprehensive analysis of 1,405 jailbreak prompts spanning from December 2022 to December 2023. We identify 131 jailbreak communities and discover unique characteristics of jailbreak prompts and their major attack strategies. We identify five highly effective jailbreak prompts that achieve 0.95 attack success rates on ChatGPT (GPT-3.5) and GPT-4.
  arXiv  Detail & Related papers  (2023-08-07T16:55:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.