Robust Critics: Defending LLMs Against Multi-Turn Attacks

• URL: http://arxiv.org/abs/2607.20472v1
• Date: Sun, 24 May 2026 05:49:14 GMT
• Title: Robust Critics: Defending LLMs Against Multi-Turn Attacks
• Abstract summary: We present Dialogue Critic Guided Sampling (DCGS) as a safety framework for multi-turn dialogue.<n>DCGS learns what the user's intent is likely to be based on the full conversational history and generates responses accordingly.<n>We prove that this inference-time reweighting approximates exponential tilting of the base policy, guaranteeing improvement in expected return for any finite candidate pool.
• Score: 36.531210410175646
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When a user asks a language model something harmful, is it a genuine attack or a misunderstood but well-meaning question? This ambiguity is one of the central challenges of LLM safety. A model that assumes the worst harms legitimate users; one that assumes the best is easily exploited. The problem is compounded in multi-turn dialogue, where an attacker's true intent may only reveal itself gradually across many exchanges, yet existing safety frameworks apply a contextual bandit treatment, ignoring the trajectory of the conversation. To that end, we propose Dialogue Critic Guided Sampling (DCGS), a framework that addresses this by inferring user intent at every turn of dialogue. Instead of applying a fixed rule about what is or is not safe, DCGS learns what the user's intent is likely to be based on the full conversational history and generates responses accordingly. Formally, we model adversarial dialogue as a Markov Decision Process and learn value and regret-based critics at both the individual token and utterance (full response) levels, scoring candidate responses via an action-value critic. We prove that this inference-time reweighting approximates exponential tilting of the base policy, guaranteeing improvement in expected return for any finite candidate pool, a property that group-relative objectives do not exhibit. Evaluated on CARES-18k, WildJailbreak, Redbench, and Harmbench, DCGS outperforms strong robust baselines and frontier models on adversarial dialogue tasks. DCGS also transfers to frontier models, improving their robustness without fine-tuning.

Related papers

• Benchmarking Factual Robustness of LLMs via Multi-conversation Persuasion [0.0]
  Red-teaming frameworks evaluate models in multi-turn dialogues where the target model retains conversation history.<n>We introduce the textbfSAST-IR (Stateful Attacker, Stateless Target - Iterative Refinement) framework.<n>We simulate a worst-case adversarial setting using textbfmulti-turn (stateless) iterations.<n>Simple, diverse attack strategies achieved a staggering textbf96% success rate, exposing severe brittleness in memory-less defense.
  arXiv  Detail & Related papers  (2026-09-15T07:46:01Z)
• SRD-GUARD: A Defense Framework of LLMs via Semantic Rewriting and Joint Multi-Model Scoring for Latent Intent Exposure [7.121327089434882]
  SRD-GUARD is a parameter-free, black-box defense framework that exposes concealed intent through semantic rewriting and consensus-based risk assessment.<n>We evaluate SRD-GUARD against UNIATTACK, CIPHER, and DeepInception on Llama-3-8B-Uncensored and DeepSeek-V4-Flash.
  arXiv  Detail & Related papers  (2026-09-06T11:26:08Z)
• One Turn Too Late: Response-Aware Defense Against Hidden Malicious Intent in Multi-Turn Dialogue [55.98008208209856]
  Hidden malicious intent in multi-turn dialogue poses a growing threat to large language models (LLMs)<n>Recent studies show that even modern commercial models with advanced guardrails remain vulnerable to such attacks despite advances in safety alignment and external guardrails.<n>In this work, we address this challenge by detecting the earliest turn at which delivering the candidate response would make the accumulated interaction sufficient to enable harmful action.
  arXiv  Detail & Related papers  (2026-05-07T03:35:31Z)
• Aligning Deep Implicit Preferences by Learning to Reason Defensively [22.548051297731416]
  We propose Critique-Driven Reasoning Alignment (CDRA) to bridge the preference inference gap.<n>CDRA reframes alignment from a scalar reward-matching task into a structured reasoning process.<n> Experiments demonstrate that CDRA excels at discovering and aligning with users' true preferences while executing robust reasoning.
  arXiv  Detail & Related papers  (2025-10-13T09:26:47Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attacks [59.300698230887114]
  Large language models (LLMs) are shown to be vulnerable to jailbreaking attacks where adversarial prompts are designed to elicit harmful responses.<n>We propose a safety steering framework grounded in safe control theory, ensuring invariant safety in multi-turn dialogues.
  arXiv  Detail & Related papers  (2025-02-28T21:10:03Z)
• Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models [53.580928907886324]
  Reasoning-Augmented Conversation is a novel multi-turn jailbreak framework.<n>It reformulates harmful queries into benign reasoning tasks.<n>We show that RACE achieves state-of-the-art attack effectiveness in complex conversational scenarios.
  arXiv  Detail & Related papers  (2025-02-16T09:27:44Z)
• JoTR: A Joint Transformer and Reinforcement Learning Framework for Dialog Policy Learning [53.83063435640911]
  Dialogue policy learning (DPL) is a crucial component of dialogue modelling. We introduce a novel framework, JoTR, to generate flexible dialogue actions. Unlike traditional methods, JoTR formulates a word-level policy that allows for a more dynamic and adaptable dialogue action generation.
  arXiv  Detail & Related papers  (2023-09-01T03:19:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.