Insider Attacks in Multi-Agent LLM Consensus Systems

• URL: http://arxiv.org/abs/2605.08268v1
• Date: Fri, 08 May 2026 03:10:22 GMT
• Title: Insider Attacks in Multi-Agent LLM Consensus Systems
• Abstract summary: We study insider manipulation in multi-agent consensus systems.<n>We propose a world-model-based framework that learns surrogate dynamics over the latent behavioral states of benign agents.<n>Preliminary results show that the trained attacker reduces the benign consensus rate and prolongs disagreement more effectively than the direct malicious-prompt baseline.
• Score: 8.207909009186091
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly deployed in multi-agent systems where agents communicate in natural language to solve tasks jointly. A key capability in such systems is consensus formation, where agents iteratively exchange messages and update decisions to reach a shared outcome. However, most existing multi-agent LLM frameworks assume that all participating agents are aligned with the system objective. In practice, a malicious insider may participate as a legitimate member of the group while pursuing a hidden adversarial goal. In this work, we study insider manipulation in multi-agent LLM consensus systems. We formalize the problem as a sequential decision-making task in which a malicious agent seeks to delay or prevent agreement among benign agents. To make attack optimization tractable, we propose a world-model-based framework that learns surrogate dynamics over the latent behavioral states of benign agents and then trains an attacker using reinforcement learning based on this learned model. Preliminary results show that the trained attacker reduces the benign consensus rate and prolongs disagreement more effectively than the direct malicious-prompt baseline. These results suggest that combining latent world models with reinforcement learning is a promising direction for adaptive insider attacks in language-based multi-agent systems.

Related papers

• MARTI-MARS$^2$: Scaling Multi-Agent Self-Search via Reinforcement Learning for Code Generation [64.2621682259008]
  Multi-Agent Reinforced Training and Inference Framework with Self-Search Scaling (MARTI-MARS2)<n>We propose a Multi-Agent Reinforced Training and Inference Framework with Self-Search Scaling (MARTI-MARS2) to integrate policy learning with multi-agent tree search.<n>We show that MARTI-MARS2 achieves 77.7%, outperforming strong baselines like GPT-5.1 on challenging code generation benchmarks.
  arXiv  Detail & Related papers  (2026-02-08T07:28:44Z)
• Multi-Agent Deep Reinforcement Learning for Collaborative UAV Relay Networks under Jamming Atatcks [36.380478794869234]
  This paper formulates this challenge as a cooperative Multi-Agent Reinforcement Learning (MARL) problem, solved using the Training with Decentralized Execution (CTDE) framework.<n>Our proposed framework significantly outperforms baselines, increasing total system throughput by approximately 50% while simultaneously achieving a near-zero collision rate.<n>A key finding is that the agents develop an emergent anti-jamming strategy without explicit programming.
  arXiv  Detail & Related papers  (2025-12-09T08:11:21Z)
• Learning Robust Social Strategies with Large Language Models [7.697496386429445]
  Reinforcement learning is effective for aligning large language models (LLMs) in the single-agent regime.<n>We show that standard RL in multi-agent settings often converges to defecting, self-interested policies.<n>To address this tendency of RL to converge to poor equilibria, we adapt a recent opponent-learning awareness algorithm, Advantage Alignment.
  arXiv  Detail & Related papers  (2025-11-24T18:43:46Z)
• AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning [78.5751183537704]
  AdvEvo-MARL is a co-evolutionary multi-agent reinforcement learning framework that internalizes safety into task agents.<n>Rather than relying on external guards, AdvEvo-MARL jointly optimize attackers and defenders.
  arXiv  Detail & Related papers  (2025-10-02T02:06:30Z)
• Can an Individual Manipulate the Collective Decisions of Multi-Agents? [53.01767232004823]
  M-Spoiler is a framework that simulates agent interactions within a multi-agent system to generate adversarial samples.<n>M-Spoiler introduces a stubborn agent that actively aids in optimizing adversarial samples.<n>Our findings confirm the risks posed by the knowledge of an individual agent in multi-agent systems.
  arXiv  Detail & Related papers  (2025-09-20T01:54:20Z)
• BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attacks [58.959622170433725]
  BlindGuard is an unsupervised defense method that learns without requiring any attack-specific labels or prior knowledge of malicious behaviors.<n>We show that BlindGuard effectively detects diverse attack types (i.e., prompt injection, memory poisoning, and tool attack) across multi-agent systems.
  arXiv  Detail & Related papers  (2025-08-11T16:04:47Z)
• Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems [0.0]
  This report addresses the early stages of risk identification and analysis for multi-agent AI systems.<n>We examine six critical failure modes: cascading reliability failures, inter-agent communication failures, monoculture collapse, conformity bias, deficient theory of mind, and mixed motive dynamics.
  arXiv  Detail & Related papers  (2025-08-06T06:06:57Z)
• How to Train a Leader: Hierarchical Reasoning in Multi-Agent LLMs [16.853362180877593]
  We introduce a hierarchical multi-agent framework that trains only a single leader LLM to coordinate a team of untrained peer agents.<n>Our results highlight the effectiveness and efficiency of training a single, flexible leader for collaborative reasoning in multi-agent LLM systems.
  arXiv  Detail & Related papers  (2025-07-11T18:34:07Z)
• AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents [54.29555239363013]
  We propose a generic black-box fuzzing framework, AgentVigil, to automatically discover and exploit indirect prompt injection vulnerabilities.<n>We evaluate AgentVigil on two public benchmarks, AgentDojo and VWA-adv, where it achieves 71% and 70% success rates against agents based on o3-mini and GPT-4o.<n>We apply our attacks in real-world environments, successfully misleading agents to navigate to arbitrary URLs, including malicious sites.
  arXiv  Detail & Related papers  (2025-05-09T07:40:17Z)
• Red-Teaming LLM Multi-Agent Systems via Communication Attacks [10.872328358364776]
  Large Language Model-based Multi-Agent Systems (LLM-MAS) have revolutionized complex problem-solving capability by enabling sophisticated agent collaboration through message-based communications.<n>We introduce Agent-in-the-Middle (AiTM), a novel attack that exploits the fundamental communication mechanisms in LLM-MAS by intercepting and manipulating inter-agent messages.
  arXiv  Detail & Related papers  (2025-02-20T18:55:39Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.