xJailbreak: Representation Space Guided Reinforcement Learning for Interpretable LLM Jailbreaking

• URL: http://arxiv.org/abs/2501.16727v2
• Date: Thu, 30 Jan 2025 16:17:56 GMT
• Title: xJailbreak: Representation Space Guided Reinforcement Learning for Interpretable LLM Jailbreaking
• Authors: Sunbowen Lee, Shiwen Ni, Chi Wei, Shuaimin Li, Liyang Fan, Ahmadreza Argha, Hamid Alinejad-Rokny, Ruifeng Xu, Yicheng Gong, Min Yang,
• Abstract summary: Black-box jailbreak is an attack where crafted prompts bypass safety mechanisms in large language models.<n>We propose a novel black-box jailbreak method leveraging reinforcement learning (RL)<n>We introduce a comprehensive jailbreak evaluation framework incorporating keywords, intent matching, and answer validation to provide a more rigorous and holistic assessment of jailbreak success.
• Score: 32.89084809038529
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Safety alignment mechanism are essential for preventing large language models (LLMs) from generating harmful information or unethical content. However, cleverly crafted prompts can bypass these safety measures without accessing the model's internal parameters, a phenomenon known as black-box jailbreak. Existing heuristic black-box attack methods, such as genetic algorithms, suffer from limited effectiveness due to their inherent randomness, while recent reinforcement learning (RL) based methods often lack robust and informative reward signals. To address these challenges, we propose a novel black-box jailbreak method leveraging RL, which optimizes prompt generation by analyzing the embedding proximity between benign and malicious prompts. This approach ensures that the rewritten prompts closely align with the intent of the original prompts while enhancing the attack's effectiveness. Furthermore, we introduce a comprehensive jailbreak evaluation framework incorporating keywords, intent matching, and answer validation to provide a more rigorous and holistic assessment of jailbreak success. Experimental results show the superiority of our approach, achieving state-of-the-art (SOTA) performance on several prominent open and closed-source LLMs, including Qwen2.5-7B-Instruct, Llama3.1-8B-Instruct, and GPT-4o-0806. Our method sets a new benchmark in jailbreak attack effectiveness, highlighting potential vulnerabilities in LLMs. The codebase for this work is available at https://github.com/Aegis1863/xJailbreak.

Related papers

• Prefill-Based Jailbreak: A Novel Approach of Bypassing LLM Safety Boundary [2.4329261266984346]
  Large Language Models (LLMs) are designed to generate helpful and safe content. adversarial attacks, commonly referred to as jailbreak, can bypass their safety protocols. We introduce a novel jailbreak attack method that leverages the prefilling feature of LLMs.
  arXiv  Detail & Related papers  (2025-04-28T07:38:43Z)
• CCJA: Context-Coherent Jailbreak Attack for Aligned Large Language Models [18.06388944779541]
  "jailbreaking" is the use of large language models to trigger unintended behaviors. We propose a novel method to balance the jailbreak attack success rate with semantic coherence. Our method is superior to state-of-the-art baselines in attack effectiveness.
  arXiv  Detail & Related papers  (2025-02-17T02:49:26Z)
• Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense [57.86886012610389]
  jailbreak attacks exploit vulnerabilities to elicit unintended or harmful outputs.<n>We introduce Layer-AdvPatcher, a novel methodology designed to defend against jailbreak attacks.<n>We conduct extensive experiments on two models, four benchmark datasets, and multiple state-of-the-art jailbreak benchmarks to demonstrate the efficacy of our approach.
  arXiv  Detail & Related papers  (2025-01-05T19:06:03Z)
• JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMs [11.924542310342282]
  We present JailPO, a novel black-box jailbreak framework to examine Large Language Models (LLMs) alignment.<n>For scalability and universality, JailPO meticulously trains attack models to automatically generate covert jailbreak prompts.<n>We also introduce a preference optimization-based attack method to enhance the jailbreak effectiveness.
  arXiv  Detail & Related papers  (2024-12-20T07:29:10Z)
• LIAR: Leveraging Alignment (Best-of-N) to Jailbreak LLMs in Seconds [98.20826635707341]
  We propose a novel jailbreak method called LIAR (LeveragIng Alignment to jailbReak)<n>We provide theoretical insights into the possibility of jailbreaking a safety-aligned model.<n>We achieve ASR comparable to the SoTA with a 10x improvement to perplexity and a Time-to-Attack measured in seconds rather than tens of hours.
  arXiv  Detail & Related papers  (2024-12-06T18:02:59Z)
• SQL Injection Jailbreak: A Structural Disaster of Large Language Models [71.55108680517422]
  In this paper, we introduce a novel jailbreak method, which induces large language models (LLMs) to produce harmful content. By injecting jailbreak information into user prompts, SIJ successfully induces the model to output harmful content. We propose a simple defense method called Self-Reminder-Key to counter SIJ and demonstrate its effectiveness.
  arXiv  Detail & Related papers  (2024-11-03T13:36:34Z)
• Deciphering the Chaos: Enhancing Jailbreak Attacks via Adversarial Prompt Translation [71.92055093709924]
  We propose a novel method that "translates" garbled adversarial prompts into coherent and human-readable natural language adversarial prompts.<n>It also offers a new approach to discovering effective designs for jailbreak prompts, advancing the understanding of jailbreak attacks.<n>Our method achieves over 90% attack success rates against Llama-2-Chat models on AdvBench, despite their outstanding resistance to jailbreak attacks.
  arXiv  Detail & Related papers  (2024-10-15T06:31:04Z)
• EnJa: Ensemble Jailbreak on Large Language Models [69.13666224876408]
  Large Language Models (LLMs) are increasingly being deployed in safety-critical applications. LLMs can still be jailbroken by carefully crafted malicious prompts, producing content that violates policy regulations. We propose a novel EnJa attack to hide harmful instructions using prompt-level jailbreak, boost the attack success rate using a gradient-based attack, and connect the two types of jailbreak attacks via a template-based connector.
  arXiv  Detail & Related papers  (2024-08-07T07:46:08Z)
• Virtual Context: Enhancing Jailbreak Attacks with Special Token Injection [54.05862550647966]
  This paper introduces Virtual Context, which leverages special tokens, previously overlooked in LLM security, to improve jailbreak attacks. Comprehensive evaluations show that Virtual Context-assisted jailbreak attacks can improve the success rates of four widely used jailbreak methods by approximately 40%.
  arXiv  Detail & Related papers  (2024-06-28T11:35:54Z)
• Jailbreaking Large Language Models Through Alignment Vulnerabilities in Out-of-Distribution Settings [57.136748215262884]
  We introduce ObscurePrompt for jailbreaking LLMs, inspired by the observed fragile alignments in Out-of-Distribution (OOD) data.<n>We first formulate the decision boundary in the jailbreaking process and then explore how obscure text affects LLM's ethical decision boundary.<n>Our approach substantially improves upon previous methods in terms of attack effectiveness, maintaining efficacy against two prevalent defense mechanisms.
  arXiv  Detail & Related papers  (2024-06-19T16:09:58Z)
• Distract Large Language Models for Automatic Jailbreak Attack [8.364590541640482]
  We propose a novel black-box jailbreak framework for automated red teaming of Large language models. We designed malicious content concealing and memory reframing with an iterative optimization algorithm to jailbreak LLMs.
  arXiv  Detail & Related papers  (2024-03-13T11:16:43Z)
• Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreak [26.741029482196534]
  "Jailbreak Attack" is phenomenon where Large Language Models (LLMs) generate harmful responses when faced with malicious instructions. We introduce a novel automatic jailbreak method RADIAL, which bypasses the security mechanism by amplifying the potential of LLMs to generate affirmation responses. Our method achieves excellent attack performance on English malicious instructions with five open-source advanced LLMs while maintaining robust attack performance in executing cross-language attacks against Chinese malicious instructions.
  arXiv  Detail & Related papers  (2023-12-07T08:29:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.