Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification

• URL: http://arxiv.org/abs/2605.28104v1
• Date: Wed, 27 May 2026 07:56:33 GMT
• Title: Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification
• Abstract summary: Malicious agents in Large Language Model-based Multi-Agent Systems (MAS) may inject misinformation to mislead other agents and disrupt system performance.<n>We propose an adaptive cooperative attack framework, where malicious agents autonomously coordinate and dynamically adjust their attack strategies.<n>We introduce Sentence-Level Trustworthiness Analysis and Rectification (STAR), a defense framework that identifies and rectifies misleading information at the sentence level within agent communications.
• Score: 42.88763759237844
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent years have witnessed the rapid development of Large Language Model-based Multi-Agent Systems (MAS), which excel at collaborative decision-making and complex problem-solving. However, malicious agents in MAS may inject misinformation to mislead other agents and disrupt system performance, giving rise to a new research direction that focuses on attack mechanisms and defense strategies in MAS. Prior studies largely assume malicious agents act independently and investigate the corresponding defense strategies. However, we argue that malicious agents may exhibit collaborative behaviors, enabling more effective attacks through internal information exchange. In this paper, we propose an adaptive cooperative attack framework, where malicious agents autonomously coordinate and dynamically adjust their attack strategies through multi-round interactions. Furthermore, we introduce Sentence-Level Trustworthiness Analysis and Rectification (STAR), a defense framework that identifies and rectifies misleading information at the sentence level within agent communications. Our experiments show that cooperative attacks lead to a significantly larger degradation in task success rate than independent attacks, resulting in a relative drop of 5.34\%. Meanwhile, STAR effectively mitigates both cooperative and independent threats and improves task success rate by an average of 36.76\%. The code is available at https://github.com/smoooom/STAR.

Related papers

• Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Turn Attacks [56.45762972211923]
  We present a proactive defense framework for securing LLMs against evolving multi-turn adversarial attacks.<n>It employs a cooperative multi-agent architecture in which specialized agents execute complementary defense strategies.<n>We present the EMRA dataset designed to simulate evolving strategies across multi-turn attacks.
  arXiv  Detail & Related papers  (2026-07-31T14:04:49Z)
• GroupGuard: A Framework for Modeling and Defending Collusive Attacks in Multi-Agent Systems [12.188883494965694]
  GroupGuard is a training-free defense framework that employs a multi-layered defense strategy to identify and isolate collusive agents.<n>GroupGuard consistently achieves high detection accuracy (up to 88%) and effectively restores collaborative performance.
  arXiv  Detail & Related papers  (2026-03-14T13:19:13Z)
• Adversarial Reinforcement Learning for Large Language Model Agent Safety [20.704989548285372]
  Large Language Model (LLM) agents can leverage tools like Google Search to complete complex tasks.<n>Current defense strategies rely on fine-tuning LLM agents on datasets of known attacks.<n>We propose Adversarial Reinforcement Learning for Agent Safety (ARLAS), a novel framework that leverages adversarial reinforcement learning (RL) by formulating the problem as a two-player zero-sum game.
  arXiv  Detail & Related papers  (2025-10-06T23:09:18Z)
• AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning [78.5751183537704]
  AdvEvo-MARL is a co-evolutionary multi-agent reinforcement learning framework that internalizes safety into task agents.<n>Rather than relying on external guards, AdvEvo-MARL jointly optimize attackers and defenders.
  arXiv  Detail & Related papers  (2025-10-02T02:06:30Z)
• Can an Individual Manipulate the Collective Decisions of Multi-Agents? [53.01767232004823]
  M-Spoiler is a framework that simulates agent interactions within a multi-agent system to generate adversarial samples.<n>M-Spoiler introduces a stubborn agent that actively aids in optimizing adversarial samples.<n>Our findings confirm the risks posed by the knowledge of an individual agent in multi-agent systems.
  arXiv  Detail & Related papers  (2025-09-20T01:54:20Z)
• The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration [72.33801123508145]
  Large language models (LLMs) are integral to multi-agent systems.<n>Privacy risks emerge that extend beyond memorization, direct inference, or single-turn evaluations.<n>In particular, seemingly innocuous responses, when composed across interactions, can cumulatively enable adversaries to recover sensitive information.
  arXiv  Detail & Related papers  (2025-09-16T16:57:25Z)
• Searching for Privacy Risks in LLM Agents via Simulation [61.229785851581504]
  We present a search-based framework that alternates between improving attack and defense strategies through the simulation of privacy-critical agent interactions.<n>We find that attack strategies escalate from direct requests to sophisticated tactics, such as impersonation and consent forgery.<n>The discovered attacks and defenses transfer across diverse scenarios and backbone models, demonstrating strong practical utility for building privacy-aware agents.
  arXiv  Detail & Related papers  (2025-08-14T17:49:09Z)
• Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influence [41.14664289570607]
  Adrial Minority Influence (AMI) is a practical black-box attack and can be launched without knowing victim parameters. AMI is also strong by considering the complex multi-agent interaction and the cooperative goal of agents. We achieve the first successful attack against real-world robot swarms and effectively fool agents in simulated environments into collectively worst-case scenarios.
  arXiv  Detail & Related papers  (2023-02-07T08:54:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.