Siren: A Learning-Based Multi-Turn Attack Framework for Simulating Real-World Human Jailbreak Behaviors

• URL: http://arxiv.org/abs/2501.14250v1
• Date: Fri, 24 Jan 2025 05:31:27 GMT
• Title: Siren: A Learning-Based Multi-Turn Attack Framework for Simulating Real-World Human Jailbreak Behaviors
• Abstract summary: We propose a learning-based multi-turn attack framework designed to simulate real-world human jailbreak behaviors.<n>Experiments demonstrate that Siren achieves an attack success rate (ASR) of 90% with LLaMA-3-8B as the attacker.<n>We hope Siren inspires the development of stronger defenses against advanced multi-turn jailbreak attacks.
• Score: 12.550678408719756
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large language models (LLMs) are widely used in real-world applications, raising concerns about their safety and trustworthiness. While red-teaming with jailbreak prompts exposes the vulnerabilities of LLMs, current efforts focus primarily on single-turn attacks, overlooking the multi-turn strategies used by real-world adversaries. Existing multi-turn methods rely on static patterns or predefined logical chains, failing to account for the dynamic strategies during attacks. We propose Siren, a learning-based multi-turn attack framework designed to simulate real-world human jailbreak behaviors. Siren consists of three stages: (1) training set construction utilizing Turn-Level LLM feedback (Turn-MF), (2) post-training attackers with supervised fine-tuning (SFT) and direct preference optimization (DPO), and (3) interactions between the attacking and target LLMs. Experiments demonstrate that Siren achieves an attack success rate (ASR) of 90% with LLaMA-3-8B as the attacker against Gemini-1.5-Pro as the target model, and 70% with Mistral-7B against GPT-4o, significantly outperforming single-turn baselines. Moreover, Siren with a 7B-scale model achieves performance comparable to a multi-turn baseline that leverages GPT-4o as the attacker, while requiring fewer turns and employing decomposition strategies that are better semantically aligned with attack goals. We hope Siren inspires the development of stronger defenses against advanced multi-turn jailbreak attacks under realistic scenarios. Code is available at https://github.com/YiyiyiZhao/siren. Warning: This paper contains potentially harmful text.

Related papers

• Fully Unleashing the Multimodal Attacker: Meta-Adaptive Jailbreaking of Vision-Language Models [25.860894697929737]
  We propose Meta-Adaptive Multimodal Jailbreaking (MAMJ) to optimize the attacker itself along two axes: an attack strategy prompt (ASP) governing attack iteration and attacker model weights determining attack effectiveness.<n>On MM-SafetyBench, MAMJ achieves 81.0%, 78.9%, and 82.3% ASR against GPT-4o, Gemini-3-Pro-adaptive, and Seed 2.0, respectively, outperforming the strongest sample-level baseline by up to 24.1 percentage points.
  arXiv  Detail & Related papers  (2026-08-27T15:23:22Z)
• DarkLLM: Learning Language-Driven Adversarial Attacks with Large Language Models [88.47231141694805]
  We present DarkLLM, a novel attack framework that trains an LLM to translate natural-language attack instructions into latent attack vectors.<n>By leveraging natural-language instruction tuning, DarkLLM unifies targeted, untargeted, segmentation, and multi-model attacks within a single framework.<n>We demonstrate that DarkLLM with only 1B parameters can follow attacker instructions and generate highly effective attacks against CLIP, SAM, and frontier LLMs.
  arXiv  Detail & Related papers  (2026-05-15T12:28:16Z)
• The Salami Slicing Threat: Exploiting Cumulative Risks in LLM Systems [25.286238948800033]
  Large Language Models (LLMs) face prominent security risks from jailbreaking.<n>We propose textitSalami Slicing Risk, which operates by chaining numerous low-risk inputs that individually evade alignment thresholds.<n>Our findings provide critical insights into the pervasive risks of multi-turn jailbreaking and offer actionable mitigation strategies.
  arXiv  Detail & Related papers  (2026-04-13T11:12:30Z)
• Multi-Turn Adaptive Prompting Attack on Large Vision-Language Models [17.22893761084286]
  Multi-turn jailbreak attacks are effective against text-only large language models (LLMs) by gradually introducing malicious content across turns.<n>When extended to large vision-language models (LVLMs), we find that naively adding visual inputs can cause existing multi-turn jailbreaks to be easily defended.<n>We propose MAPA: a multi-turn adaptive prompting attack that alternates text-vision attack actions to elicit the most malicious response.
  arXiv  Detail & Related papers  (2026-02-16T02:15:58Z)
• SEMA: Simple yet Effective Learning for Multi-Turn Jailbreak Attacks [53.97948802255959]
  We propose a framework that trains a multi-turn attacker without relying on any existing strategies or external data.<n>Prefilling self-tuning enables usable rollouts by fine-tuning on non-refusal, well-structured, multi-turn adversarial prompts.<n>We anchor harmful intent in multi-turn jailbreaks via an intent-drift-aware reward that combines intent alignment, compliance risk, and level of detail.
  arXiv  Detail & Related papers  (2026-02-06T16:44:57Z)
• RL-MTJail: Reinforcement Learning for Automated Black-Box Multi-Turn Jailbreaking of Large Language Models [60.201244463046784]
  Large language models are vulnerable to jailbreak attacks.<n>This paper studies black-box multi-turn jailbreaks, aiming to train attacker LLMs to elicit harmful content from black-box models.
  arXiv  Detail & Related papers  (2025-12-08T17:42:59Z)
• MTAttack: Multi-Target Backdoor Attacks against Large Vision-Language Models [52.37749859972453]
  We propose MTAttack, the first multi-target backdoor attack framework for enforcing accurate multiple trigger-target mappings in LVLMs.<n> Experiments on popular benchmarks demonstrate a high success rate of MTAttack for multi-target attacks.<n>Our attack exhibits strong generalizability across datasets and robustness against backdoor defense strategies.
  arXiv  Detail & Related papers  (2025-11-13T09:00:21Z)
• PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploits [0.12744523252873352]
  PLAGUE is a plug-and-play framework for designing multi-turn attacks inspired by lifelong-learning agents.<n>We show that PLAGUE achieves state-of-the-art jailbreaking results, improving attack success rates (ASR) by more than 30% across leading models.
  arXiv  Detail & Related papers  (2025-10-20T17:37:03Z)
• A Dynamic Stackelberg Game Framework for Agentic AI Defense Against LLM Jailbreaking [13.343937277604892]
  This paper presents a Stackelberg game framework to model the interactions between attackers and defenders in the context of large language models jailbreaking.<n>We propose a novel agentic AI solution, the "Purple Agent," which integrates adversarial exploration and defensive strategies using Rapidly-exploring Random Trees (RRT)
  arXiv  Detail & Related papers  (2025-07-10T22:37:47Z)
• Lifelong Safety Alignment for Language Models [33.90238075760236]
  We propose a lifelong safety alignment framework for jailbreaking defenses.<n>A Meta-Attacker is trained to actively discover novel jailbreaking strategies, and a Defender is trained to resist them.<n>Our framework reduces the Meta-Attacker's success rate to just 7%, enabling safer and more reliable deployment of LLMs.
  arXiv  Detail & Related papers  (2025-05-26T17:40:40Z)
• Strategize Globally, Adapt Locally: A Multi-Turn Red Teaming Agent with Dual-Level Learning [39.931442440365444]
  AlgName is a novel red-teaming agent that emulates sophisticated human attackers through complementary learning dimensions. AlgName enables the agent to identify new jailbreak tactics, develop a goal-based tactic selection framework, and refine prompt formulations for selected tactics. Empirical evaluations on JailbreakBench demonstrate our framework's superior performance, achieving over 90% attack success rates against GPT-3.5-Turbo and Llama-3.1-70B within 5 conversation turns.
  arXiv  Detail & Related papers  (2025-04-02T01:06:19Z)
• One-Shot is Enough: Consolidating Multi-Turn Attacks into Efficient Single-Turn Prompts for LLMs [8.91993614197627]
  We introduce a novel approach called Multi-turn-to-Single-turn (M2S) that systematically converts multi-turn jailbreak prompts into single-turn attacks. Our experiments show that M2S often increases or maintains high Attack Success Rates (ASRs) compared to original multi-turn conversations.
  arXiv  Detail & Related papers  (2025-03-06T07:34:51Z)
• Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models [53.580928907886324]
  Reasoning-Augmented Conversation is a novel multi-turn jailbreak framework. It reformulates harmful queries into benign reasoning tasks. We show that RACE achieves state-of-the-art attack effectiveness in complex conversational scenarios.
  arXiv  Detail & Related papers  (2025-02-16T09:27:44Z)
• Deciphering the Chaos: Enhancing Jailbreak Attacks via Adversarial Prompt Translation [71.92055093709924]
  We propose a novel method that "translates" garbled adversarial prompts into coherent and human-readable natural language adversarial prompts.<n>It also offers a new approach to discovering effective designs for jailbreak prompts, advancing the understanding of jailbreak attacks.<n>Our method achieves over 90% attack success rates against Llama-2-Chat models on AdvBench, despite their outstanding resistance to jailbreak attacks.
  arXiv  Detail & Related papers  (2024-10-15T06:31:04Z)
• Derail Yourself: Multi-turn LLM Jailbreak Attack through Self-discovered Clues [88.96201324719205]
  This study exposes the safety vulnerabilities of Large Language Models (LLMs) in multi-turn interactions. We introduce ActorAttack, a novel multi-turn attack method inspired by actor-network theory.
  arXiv  Detail & Related papers  (2024-10-14T16:41:49Z)
• RED QUEEN: Safeguarding Large Language Models against Concealed Multi-Turn Jailbreaking [30.67803190789498]
  We propose a new jailbreak approach, RED QUEEN ATTACK, that constructs a multi-turn scenario, concealing the malicious intent under the guise of preventing harm. Our experiments reveal that all LLMs are vulnerable to RED QUEEN ATTACK, reaching 87.62% attack success rate on GPT-4o and 75.4% on Llama3-70B. To prioritize safety, we introduce a straightforward mitigation strategy called RED QUEEN GUARD, which aligns LLMs to effectively counter adversarial attacks.
  arXiv  Detail & Related papers  (2024-09-26T01:24:17Z)
• h4rm3l: A language for Composable Jailbreak Attack Synthesis [48.5611060845958]
  h4rm3l is a novel approach that addresses the gap with a human-readable domain-specific language. We show that h4rm3l's synthesized attacks are diverse and more successful than existing jailbreak attacks in literature.
  arXiv  Detail & Related papers  (2024-08-09T01:45:39Z)
• SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner [21.414701448926614]
  This paper introduces a generic LLM jailbreak defense framework called SelfDefend. We empirically validate using the commonly used GPT-3.5/4 models across all major jailbreak attacks. These models outperform six state-of-the-art defenses and match the performance of GPT-4-based SelfDefend.
  arXiv  Detail & Related papers  (2024-06-08T15:45:31Z)
• Adversarial Tuning: Defending Against Jailbreak Attacks for LLMs [13.317364896194903]
  We propose a two-stage adversarial tuning framework to enhance Large Language Models' generalized defense capabilities. In the first stage, we introduce the hierarchical meta-universal adversarial prompt learning to efficiently generate token-level adversarial prompts. In the second stage, we propose the automatic adversarial prompt learning to iteratively refine semantic-level adversarial prompts.
  arXiv  Detail & Related papers  (2024-06-07T15:37:15Z)
• Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attacks [59.46556573924901]
  This paper introduces Defensive Prompt Patch (DPP), a novel prompt-based defense mechanism for large language models (LLMs) Unlike previous approaches, DPP is designed to achieve a minimal Attack Success Rate (ASR) while preserving the high utility of LLMs. Empirical results conducted on LLAMA-2-7B-Chat and Mistral-7B-Instruct-v0.2 models demonstrate the robustness and adaptability of DPP.
  arXiv  Detail & Related papers  (2024-05-30T14:40:35Z)
• Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks [17.22989422489567]
  Large language models (LLMs) are vulnerable to adversarial attacks or jailbreaking. We propose an optimization-based objective for defending LLMs against jailbreaking attacks and an algorithm to create robust system-level defenses. Our results show improved robustness to both jailbreaks seen during optimization and unknown jailbreaks, reducing the attack success rate (ASR) on GPT-4 to 6% and Llama-2 to 0% on JailbreakBench.
  arXiv  Detail & Related papers  (2024-01-30T18:56:08Z)
• Weak-to-Strong Jailbreaking on Large Language Models [96.50953637783581]
  Large language models (LLMs) are vulnerable to jailbreak attacks. Existing jailbreaking methods are computationally costly. We propose the weak-to-strong jailbreaking attack.
  arXiv  Detail & Related papers  (2024-01-30T18:48:37Z)
• Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization [98.18718484152595]
  We propose to integrate goal prioritization at both training and inference stages to counteract the intrinsic conflict between the goals of being helpful and ensuring safety. Our work thus contributes to the comprehension of jailbreaking attacks and defenses, and sheds light on the relationship between LLMs' capability and safety.
  arXiv  Detail & Related papers  (2023-11-15T16:42:29Z)
• SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks [99.23352758320945]
  We propose SmoothLLM, the first algorithm designed to mitigate jailbreaking attacks on large language models (LLMs) Based on our finding that adversarially-generated prompts are brittle to character-level changes, our defense first randomly perturbs multiple copies of a given input prompt, and then aggregates the corresponding predictions to detect adversarial inputs.
  arXiv  Detail & Related papers  (2023-10-05T17:01:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.