CoopGuard: Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Round Attacks

• URL: http://arxiv.org/abs/2604.04060v1
• Date: Sun, 05 Apr 2026 11:06:13 GMT
• Title: CoopGuard: Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Round Attacks
• Abstract summary: CoopGuard is a stateful multi-round LLM defense framework based on cooperative agents.<n>It employs three specialized agents for complementary round-level strategies.<n> Experiments show that CoopGuard reduces attack success rate by 78.9% over state-of-the-art defenses.
• Score: 31.582651893036683
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As Large Language Models (LLMs) are increasingly deployed in complex applications, their vulnerability to adversarial attacks raises urgent safety concerns, especially those evolving over multi-round interactions. Existing defenses are largely reactive and struggle to adapt as adversaries refine strategies across rounds. In this work, we propose CoopGuard , a stateful multi-round LLM defense framework based on cooperative agents that maintains and updates an internal defense state to counter evolving attacks. It employs three specialized agents (Deferring Agent, Tempting Agent, and Forensic Agent) for complementary round-level strategies, coordinated by System Agent, which conditions decisions on the evolving defense state (interaction history) and orchestrates agents over time. To evaluate evolving threats, we introduce the EMRA benchmark with 5,200 adversarial samples across 8 attack types, simulating progressively LLM multi-round attacks. Experiments show that CoopGuard reduces attack success rate by 78.9% over state-of-the-art defenses, while improving deceptive rate by 186% and reducing attack efficiency by 167.9%, offering a more comprehensive assessment of multi-round defense. These results demonstrate that CoopGuard provides robust protection for LLMs in multi-round adversarial scenarios.

Related papers

• CoER: Defending against Adaptive Indirect Prompt Injection via Adversarial Co-Evolution and Refinement [19.128257554975104]
  Language-model agents are vulnerable to indirect prompt injection (IPI) during tool use.<n>We propose CoER, a verifier-grounded co-evolution and refinement framework.<n>In our main seven-domain evaluation, CoER reduces observed overall attack success from 38.5% to 0.2%.
  arXiv  Detail & Related papers  (2026-09-07T14:09:11Z)
• Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Turn Attacks [56.45762972211923]
  We present a proactive defense framework for securing LLMs against evolving multi-turn adversarial attacks.<n>It employs a cooperative multi-agent architecture in which specialized agents execute complementary defense strategies.<n>We present the EMRA dataset designed to simulate evolving strategies across multi-turn attacks.
  arXiv  Detail & Related papers  (2026-07-31T14:04:49Z)
• Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification [42.88763759237844]
  Malicious agents in Large Language Model-based Multi-Agent Systems (MAS) may inject misinformation to mislead other agents and disrupt system performance.<n>We propose an adaptive cooperative attack framework, where malicious agents autonomously coordinate and dynamically adjust their attack strategies.<n>We introduce Sentence-Level Trustworthiness Analysis and Rectification (STAR), a defense framework that identifies and rectifies misleading information at the sentence level within agent communications.
  arXiv  Detail & Related papers  (2026-05-27T07:56:33Z)
• GroupGuard: A Framework for Modeling and Defending Collusive Attacks in Multi-Agent Systems [12.188883494965694]
  GroupGuard is a training-free defense framework that employs a multi-layered defense strategy to identify and isolate collusive agents.<n>GroupGuard consistently achieves high detection accuracy (up to 88%) and effectively restores collaborative performance.
  arXiv  Detail & Related papers  (2026-03-14T13:19:13Z)
• AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning [78.5751183537704]
  AdvEvo-MARL is a co-evolutionary multi-agent reinforcement learning framework that internalizes safety into task agents.<n>Rather than relying on external guards, AdvEvo-MARL jointly optimize attackers and defenders.
  arXiv  Detail & Related papers  (2025-10-02T02:06:30Z)
• Evo-MARL: Co-Evolutionary Multi-Agent Reinforcement Learning for Internalized Safety [54.228018540152924]
  Multi-agent systems (MAS) built on multimodal large language models exhibit strong collaboration and performance.<n>Evo-MARL is a novel multi-agent reinforcement learning framework that enables all task agents to jointly acquire defensive capabilities.<n>Evo-MARL reduces attack success rates by up to 22% while boosting accuracy by up to 5% on reasoning tasks.
  arXiv  Detail & Related papers  (2025-08-05T19:26:55Z)
• Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models [64.47869632167284]
  Conventional language model (LM) safety alignment relies on a reactive, disjoint procedure: attackers exploit a static model, followed by defensive fine-tuning to patch exposed vulnerabilities.<n>This sequential approach creates a mismatch -- attackers overfit to obsolete defenses, while defenders perpetually lag behind emerging threats.<n>We propose Self-RedTeam, an online self-play reinforcement learning algorithm where an attacker and defender agent co-evolve through continuous interaction.
  arXiv  Detail & Related papers  (2025-06-09T06:35:12Z)
• AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security [74.22452069013289]
  AegisLLM is a cooperative multi-agent defense against adversarial attacks and information leakage.<n>We show that scaling agentic reasoning system at test-time substantially enhances robustness without compromising model utility.<n> Comprehensive evaluations across key threat scenarios, including unlearning and jailbreaking, demonstrate the effectiveness of AegisLLM.
  arXiv  Detail & Related papers  (2025-04-29T17:36:05Z)
• Large Language Model Sentinel: LLM Agent for Adversarial Purification [27.742161175314635]
  Large language models (LLMs) are vulnerable to adversarial attacks by some well-designed textual perturbations.<n>We introduce a novel defense technique named Large LAnguage MOdel Sentinel (LLAMOS) to enhance the adversarial robustness of LLMs.
  arXiv  Detail & Related papers  (2024-05-24T07:23:56Z)
• BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning [85.2564206440109]
  This paper reveals the threats in this practical scenario that backdoor attacks can remain effective even after defenses. We introduce the emphtoolns attack, which is resistant to backdoor detection and model fine-tuning defenses.
  arXiv  Detail & Related papers  (2023-11-20T02:21:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.