ContextualJailbreak: Evolutionary Red-Teaming via Simulated Conversational Priming

• URL: http://arxiv.org/abs/2605.02647v1
• Date: Mon, 04 May 2026 14:32:40 GMT
• Title: ContextualJailbreak: Evolutionary Red-Teaming via Simulated Conversational Priming
• Abstract summary: Large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safety alignment and elicit harmful responses.<n>We present ContextualJailbreak, a black-box red-teaming strategy that performs evolutionary search over a simulated multi-turn primed dialogue.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safety alignment and elicit harmful responses. A growing body of work shows that contextual priming, where earlier turns covertly bias later replies, constitutes a powerful attack surface, with hand-crafted multi-turn scaffolds consistently outperforming single-turn manipulations on capable models. However, automated optimization-based red-teaming has remained largely limited to the single-turn setting, iterating over static prompts and lacking the ability to reason about which forms of conversational priming induce compliance. While recent multi-turn, search-based approaches have begun to bridge this gap, the mutator design space underlying effective primed dialogues remains largely unexplored. We present ContextualJailbreak, a black-box red-teaming strategy that performs evolutionary search over a simulated multi-turn primed dialogue. The strategy leverages a graded 0-5 harm score from a two-level judge as an in-loop signal, enabling partially harmful responses to guide the search process rather than being discarded. Search is driven by five semantically defined mutation operators: roleplay, scenario, expand, troubleshooting, and mechanistic, of which the last two are novel contributions of this work. Across 50 representative HarmBench behaviors, ContextualJailbreak achieves an ASR of 100% on gpt-oss:20B, 100% on qwen3-8B, 100% on llama3.1:70B, and 90% on gpt-oss:120B, outperforming four single- and multi-turn baselines by 31-96 percentage points on average. The 40 maximally harmful attacks discovered against gpt-oss:120B transfer without adaptation to closed frontier models, achieving 90.0% on gpt-4o-mini, 70.0% on gpt-5, and 70.0% on gemini-3-flash, but only 17.5% on claude-opus-4-7 and 15.0% on claude-sonnet-4-6, revealing a pronounced provider-level asymmetry in alignment robustness.

Related papers

• EvoFlint: An Evolutionary Atlas of Multi-Turn LLM Vulnerabilities [21.962003812448472]
  We introduce EvoFlint, which applies evolutionary quality-diversity search to multi-turn red-teaming.<n>A risk-indexed archive runs novelty search with local competition over strategy description embeddings inside each cell.<n>The resulting archive, organized by risk category, exposes for each target which categories of harm its safety training has and has not covered.
  arXiv  Detail & Related papers  (2026-08-31T23:33:33Z)
• TRIS: A Tri-Layer Retrieval Integrity Sieve Against Knowledge Poisoning [6.613886785198943]
  Tri-Layer Sieve sanitizes retrieved evidence through cross-embedding-space clustering.<n>It exploits a key weakness of retrieval-stage poisoning: a single document must satisfy one embedding geometry, one internal Trigger-Payload structure, and one generation objective.
  arXiv  Detail & Related papers  (2026-08-31T23:21:27Z)
• Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search [50.16356451328644]
  Proving Shannon-type entropy inequalities is a fundamental task in information theory.<n>We investigate whether small-scale large language models, fine-tuned on atomic proof steps, can automate this process.<n>We show that GPT-5.5 solves 1.7% samples under zero-shot prompting while Psitip solves 33.3% samples.
  arXiv  Detail & Related papers  (2026-06-04T05:43:12Z)
• Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization [51.11398557420066]
  We introduce Metis, a framework that reformulates jailbreaking as inference-time policy optimization.<n>Metis achieves the strongest average Attack Success Rate (ASR) among compared methods at 89.2%.<n>By replacing redundant exploration with directed optimization, Metis reduces token costs by an average of 8.2x and up to 11.4x.
  arXiv  Detail & Related papers  (2026-05-11T06:45:00Z)
• MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agents [2.1942030377331245]
  Coding agents often pass per-prompt safety review yet ship exploitable code when their tasks are decomposed into routine engineering tickets.<n>We introduce MOSAIC-Bench, a benchmark of 199 three-stage attack chains paired with deterministic exploit oracles on deployed software substrates.<n>We show that nine production coding agents compose innocuous tickets at 53-86% end-to-end ASR with only two refusals across all staged runs.
  arXiv  Detail & Related papers  (2026-05-05T16:38:23Z)
• Re-Mask and Redirect: Exploiting Denoising Irreversibility in Diffusion Language Models [0.0]
  Safety alignment in diffusion language models (dLLMs) relies on a single load-bearing assumption: committed tokens are permanent.<n>We show that by re-masking committed refusal tokens and injecting a short affirmative prefix, achieves 74-82% ASR on HarmBench.<n>We call this attack TrajHijack; it is the first trajectory-level attack on dLLMs, requires no computation, and generalizes across SFT and preference-optimized (VRPO) models.
  arXiv  Detail & Related papers  (2026-03-17T02:24:37Z)
• SEMA: Simple yet Effective Learning for Multi-Turn Jailbreak Attacks [53.97948802255959]
  We propose a framework that trains a multi-turn attacker without relying on any existing strategies or external data.<n>Prefilling self-tuning enables usable rollouts by fine-tuning on non-refusal, well-structured, multi-turn adversarial prompts.<n>We anchor harmful intent in multi-turn jailbreaks via an intent-drift-aware reward that combines intent alignment, compliance risk, and level of detail.
  arXiv  Detail & Related papers  (2026-02-06T16:44:57Z)
• AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models [2.6799007584079884]
  AutoAdv is a training-free framework for automated multi-turn jailbreaking.<n>It achieves up to 95% attack success rate on Llama-3.1-8B within six turns.
  arXiv  Detail & Related papers  (2025-11-04T08:56:28Z)
• Echoes of Human Malice in Agents: Benchmarking LLMs for Multi-Turn Online Harassment Attacks [10.7231991032233]
  Large Language Model (LLM) agents are powering a growing share of interactive web applications, yet remain vulnerable to misuse and harm.<n>We present the Online Harassment Agentic Benchmark consisting of: (i) a synthetic multi-turn harassment conversation dataset, (ii) a multi-agent (e.g., harasser, victim) simulation informed by repeated game theory, (iii) three jailbreak methods attacking agents across memory, planning, and fine-tuning, and (iv) a mixed-methods evaluation framework.
  arXiv  Detail & Related papers  (2025-10-16T01:27:44Z)
• X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents [76.66287343106204]
  X-Teaming is a framework that explores how seemingly harmless interactions escalate into harmful outcomes and generates corresponding attack scenarios.<n>X-Teaming achieves a 96.2% attack success rate against the latest Claude 3.7 Sonnet model.<n>XGuard-Train is an open-source multi-turn safety training dataset that is 20x larger than the previous best resource.
  arXiv  Detail & Related papers  (2025-04-15T16:11:28Z)
• Benchmarking Reasoning Robustness in Large Language Models [76.79744000300363]
  We find significant performance degradation on novel or incomplete data.<n>These findings highlight the reliance on recall over rigorous logical inference.<n>This paper introduces a novel benchmark, termed as Math-RoB, that exploits hallucinations triggered by missing information to expose reasoning gaps.
  arXiv  Detail & Related papers  (2025-03-06T15:36:06Z)
• Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models [53.580928907886324]
  Reasoning-Augmented Conversation is a novel multi-turn jailbreak framework.<n>It reformulates harmful queries into benign reasoning tasks.<n>We show that RACE achieves state-of-the-art attack effectiveness in complex conversational scenarios.
  arXiv  Detail & Related papers  (2025-02-16T09:27:44Z)
• Scalable Reinforcement Post-Training Beyond Static Human Prompts: Evolving Alignment via Asymmetric Self-Play [52.3079697845254]
  eva is the first method that allows language models to adaptively create training prompts in both offline and online RL post-training.<n>We show eva can create effective RL curricula and is robust across ablations.
  arXiv  Detail & Related papers  (2024-10-31T08:15:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.