EvoFlint: An Evolutionary Atlas of Multi-Turn LLM Vulnerabilities

• URL: http://arxiv.org/abs/2609.00487v1
• Date: Mon, 31 Aug 2026 23:33:33 GMT
• Title: EvoFlint: An Evolutionary Atlas of Multi-Turn LLM Vulnerabilities
• Abstract summary: We introduce EvoFlint, which applies evolutionary quality-diversity search to multi-turn red-teaming.<n>A risk-indexed archive runs novelty search with local competition over strategy description embeddings inside each cell.<n>The resulting archive, organized by risk category, exposes for each target which categories of harm its safety training has and has not covered.
• Score: 21.962003812448472
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Frontier language models that refuse harmful single-turn prompts often comply when the same intent is reached gradually over many turns, making multi-turn attacks one of the least understood failure modes of large language models. Most automated red-teaming methods treat this as a generation problem: produce attacks that break the model. We argue it is better framed as a search problem: discover, organize, and iteratively refine a diverse archive of attack strategies, producing a structured map of how a target model fails rather than a list of one-off successes. We introduce EvoFlint, which applies evolutionary quality-diversity search to multi-turn red-teaming. Attack strategies are phased conversation plans, not raw prompts, and are evolved through LLM-driven mutation and crossover. A Pareto fitness over attack success rate and peak severity preserves selection signal from near-miss attacks. A risk-indexed archive runs novelty search with local competition over strategy description embeddings inside each cell, maintaining diversity without committing to a predefined style taxonomy. A generation-level memory accumulates target-model insights across the population and feeds them back into strategy generation. On the HarmBench-test split, EvoFlint reaches attack success rates of 35.8% on Claude Sonnet 4.6, 59.7% on GPT-5.4, and 94.3% on Qwen3-32B, alongside 98.7% on the older GPT-4o included as a baseline reference. The resulting archive, organized by risk category, exposes for each target which categories of harm its safety training has and has not covered.

Related papers

• Quality-Diversity Evolution for Discovering Diverse Vulnerabilities in LLM Safety [0.0]
  We introduce a quality-diversity evolutionary framework that operates at the semantic level.<n>We maintain a diverse archive of attacks across behavioral dimensions.<n>The semantic representation produces interpretable attacks that reveal systematic, model-specific weaknesses.
  arXiv  Detail & Related papers  (2026-05-30T16:40:24Z)
• ContextualJailbreak: Evolutionary Red-Teaming via Simulated Conversational Priming [0.0]
  Large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safety alignment and elicit harmful responses.<n>We present ContextualJailbreak, a black-box red-teaming strategy that performs evolutionary search over a simulated multi-turn primed dialogue.
  arXiv  Detail & Related papers  (2026-05-04T14:32:40Z)
• SEMA: Simple yet Effective Learning for Multi-Turn Jailbreak Attacks [53.97948802255959]
  We propose a framework that trains a multi-turn attacker without relying on any existing strategies or external data.<n>Prefilling self-tuning enables usable rollouts by fine-tuning on non-refusal, well-structured, multi-turn adversarial prompts.<n>We anchor harmful intent in multi-turn jailbreaks via an intent-drift-aware reward that combines intent alignment, compliance risk, and level of detail.
  arXiv  Detail & Related papers  (2026-02-06T16:44:57Z)
• OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMs [36.57820295876294]
  We introduce OpenRT, a unified, modular, and high- throughput red-teaming framework for MLLM safety evaluation.<n>At its core, OpenRT architects a paradigm shift in automated red-teaming by introducing an adversarial kernel that enables modular separation across five dimensions.<n>Our framework integrates 37 diverse attack methodologies, spanning white-box gradients, multi-modal perturbations, and sophisticated multi-agent evolutionary strategies.
  arXiv  Detail & Related papers  (2026-01-04T16:41:33Z)
• Automating Deception: Scalable Multi-Turn LLM Jailbreaks [0.7212939068975618]
  Multi-turn conversational attacks, which leverage psychological principles like Foot-in-the-Door (FITD), pose a persistent threat to Large Language Models (LLMs)<n>This paper introduces a novel, automated pipeline for generating large-scale, psychologically-grounded multi-turn jailbreak datasets.<n>We evaluate seven models from three major LLM families under both multi-turn (with history) and single-turn (without history) conditions.
  arXiv  Detail & Related papers  (2025-11-24T03:15:11Z)
• Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming Attacks [63.803415430308114]
  Current large language models are vulnerable to adversarial attacks in multi-turn interaction settings.<n>We propose DialTree-RPO, an on-policy reinforcement learning framework integrated with tree search.<n>Our approach achieves more than 25.9% higher ASR across 10 target models compared to previous state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-10-02T17:57:05Z)
• Active Attacks: Red-teaming LLMs via Adaptive Environments [71.55110023234376]
  We address the challenge of generating diverse attack prompts for large language models (LLMs)<n>We introduce textitActive Attacks, a novel RL-based red-teaming algorithm that adapts its attacks as the victim evolves.
  arXiv  Detail & Related papers  (2025-09-26T06:27:00Z)
• Advancing Jailbreak Strategies: A Hybrid Approach to Exploiting LLM Vulnerabilities and Bypassing Modern Defenses [4.706534644850809]
  Two primary inference-phase threats are token-level and prompt-level jailbreaks.<n>We propose two hybrid approaches that integrate token- and prompt-level techniques to enhance jailbreak effectiveness across diverse PTLMs.
  arXiv  Detail & Related papers  (2025-06-27T07:26:33Z)
• No Query, No Access [50.18709429731724]
  We introduce the textbfVictim Data-based Adrial Attack (VDBA), which operates using only victim texts.<n>To prevent access to the victim model, we create a shadow dataset with publicly available pre-trained models and clustering methods.<n>Experiments on the Emotion and SST5 datasets show that VDBA outperforms state-of-the-art methods, achieving an ASR improvement of 52.08%.
  arXiv  Detail & Related papers  (2025-05-12T06:19:59Z)
• RED QUEEN: Safeguarding Large Language Models against Concealed Multi-Turn Jailbreaking [28.95203269961824]
  We propose a new jailbreak approach, RED QUEEN ATTACK, that constructs a multi-turn scenario, concealing the malicious intent under the guise of preventing harm.<n>Our experiments reveal that all LLMs are vulnerable to RED QUEEN ATTACK, reaching 87.62% attack success rate on GPT-4o and 75.4% on Llama3-70B.<n>To prioritize safety, we introduce a straightforward mitigation strategy called RED QUEEN GUARD, which aligns LLMs to effectively counter adversarial attacks.
  arXiv  Detail & Related papers  (2024-09-26T01:24:17Z)
• Learning diverse attacks on large language models for robust red-teaming and safety tuning [126.32539952157083]
  Red-teaming, or identifying prompts that elicit harmful responses, is a critical step in ensuring the safe deployment of large language models.<n>We show that even with explicit regularization to favor novelty and diversity, existing approaches suffer from mode collapse or fail to generate effective attacks.<n>We propose to use GFlowNet fine-tuning, followed by a secondary smoothing phase, to train the attacker model to generate diverse and effective attack prompts.
  arXiv  Detail & Related papers  (2024-05-28T19:16:17Z)
• Improved Generation of Adversarial Examples Against Safety-aligned LLMs [72.38072942860309]
  Adversarial prompts generated using gradient-based methods exhibit outstanding performance in performing automatic jailbreak attacks against safety-aligned LLMs. In this paper, we explore a new perspective on this problem, suggesting that it can be alleviated by leveraging innovations inspired in transfer-based attacks. We show that 87% of the query-specific adversarial suffixes generated by the developed combination can induce Llama-2-7B-Chat to produce the output that exactly matches the target string on AdvBench.
  arXiv  Detail & Related papers  (2024-05-28T06:10:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.