Emergent Social Intelligence Risks in Generative Multi-Agent Systems

• URL: http://arxiv.org/abs/2603.27771v2
• Date: Sat, 04 Apr 2026 07:45:49 GMT
• Title: Emergent Social Intelligence Risks in Generative Multi-Agent Systems
• Abstract summary: Multi-agent systems are rapidly moving from lab prototypes to real-world deployments, where they jointly plan, negotiate, and allocate resources to solve complex tasks.<n>Here, we present a pioneer study of such emergent multi-agent risk in pathologies that involve competition over shared resources, sequential handoff collaboration, collective decision aggregation, and others.<n>We observe that such group behaviors arise frequently across repeated trials and a wide range of interaction conditions, rather than as rare or pathological cases.<n>These findings expose the dark side of intelligent multi-agent systems: a social intelligence risk where agent collectives spontaneously reproduce familiar failure patterns from human societies.
• Score: 106.60961969939676
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent systems composed of large generative models are rapidly moving from laboratory prototypes to real-world deployments, where they jointly plan, negotiate, and allocate shared resources to solve complex tasks. While such systems promise unprecedented scalability and autonomy, their collective interaction also gives rise to failure modes that cannot be reduced to individual agents. Understanding these emergent risks is therefore critical. Here, we present a pioneer study of such emergent multi-agent risk in workflows that involve competition over shared resources (e.g., computing resources or market share), sequential handoff collaboration (where downstream agents see only predecessor outputs), collective decision aggregation, and others. Across these settings, we observe that such group behaviors arise frequently across repeated trials and a wide range of interaction conditions, rather than as rare or pathological cases. In particular, phenomena such as collusion-like coordination and conformity emerge with non-trivial frequency under realistic resource constraints, communication protocols, and role assignments, mirroring well-known pathologies in human societies despite no explicit instruction. Moreover, these risks cannot be prevented by existing agent-level safeguards alone. These findings expose the dark side of intelligent multi-agent systems: a social intelligence risk where agent collectives, despite no instruction to do so, spontaneously reproduce familiar failure patterns from human societies.

Related papers

• Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems [65.90791804095561]
  Multi-agent systems address this through structured collaboration among specialized agents, but tighter coordination amplifies a less explored risk.<n>Existing surveys cover individual agent capabilities, multi-agent collaboration, or agent self-evolution separately, leaving the causal dependencies among them unexamined.<n>This survey provides a unified review organized around four causally linked stages, which we term the LIFE progression: Lay the capability foundation, Integrate agents through collaboration, Find faults through attribution, and Evolve through autonomous self-improvement.
  arXiv  Detail & Related papers  (2026-05-14T14:36:13Z)
• Position: AI Agents Are Not (Yet) a Panacea for Social Simulation [62.891898598784415]
  Recent advances in large language models (LLMs) have spurred growing interest in using LLM-integrated agents for social simulation.<n>This position paper argues that LLM-based agents are not (yet) a panacea for social simulation.
  arXiv  Detail & Related papers  (2026-02-19T06:35:07Z)
• Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems [0.0]
  This report addresses the early stages of risk identification and analysis for multi-agent AI systems.<n>We examine six critical failure modes: cascading reliability failures, inter-agent communication failures, monoculture collapse, conformity bias, deficient theory of mind, and mixed motive dynamics.
  arXiv  Detail & Related papers  (2025-08-06T06:06:57Z)
• SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agents [58.21223208538351]
  This work explores the security issues surrounding mobile multimodal agents.<n>It attempts to construct a risk discrimination mechanism by incorporating behavioral sequence information.<n>It also designs an automated assisted assessment scheme based on a large language model.
  arXiv  Detail & Related papers  (2025-07-01T15:10:00Z)
• Emergent Risk Awareness in Rational Agents under Resource Constraints [2.69407449467596]
  This work aims to increase understanding and interpretability of emergent behaviours of AI agents operating under survival pressure.<n>We provide theoretical and empirical results that quantify the impact of survival-driven preference shifts.<n>We propose mechanisms to mitigate the emergence of risk-seeking or risk-averse behaviours.
  arXiv  Detail & Related papers  (2025-05-29T13:31:12Z)
• The Problem of Algorithmic Collisions: Mitigating Unforeseen Risks in a Connected World [2.8775022881551666]
  The increasing deployment of Artificial Intelligence (AI) and other autonomous algorithmic systems presents the world with new systemic risks.<n>Current governance frameworks are inadequate as they lack visibility into this complex ecosystem of interactions.<n>This paper outlines the nature of this challenge and proposes some initial policy suggestions centered on increasing transparency and accountability through phased system registration, a licensing framework for deployment, and enhanced monitoring capabilities.
  arXiv  Detail & Related papers  (2025-05-26T16:22:18Z)
• Multi-Agent Risks from Advanced AI [90.74347101431474]
  Multi-agent systems of advanced AI pose novel and under-explored risks.<n>We identify three key failure modes based on agents' incentives, as well as seven key risk factors.<n>We highlight several important instances of each risk, as well as promising directions to help mitigate them.
  arXiv  Detail & Related papers  (2025-02-19T23:03:21Z)
• Safe Multi-agent Learning via Trapping Regions [89.24858306636816]
  We apply the concept of trapping regions, known from qualitative theory of dynamical systems, to create safety sets in the joint strategy space for decentralized learning. We propose a binary partitioning algorithm for verification that candidate sets form trapping regions in systems with known learning dynamics, and a sampling algorithm for scenarios where learning dynamics are not known.
  arXiv  Detail & Related papers  (2023-02-27T14:47:52Z)
• Learning Collective Action under Risk Diversity [68.88688248278102]
  We investigate the consequences of risk diversity in groups of agents learning to play collective risk dilemmas. We show that risk diversity significantly reduces overall cooperation and hinders collective target achievement. Our results highlight the need for aligning risk perceptions among agents or develop new learning techniques.
  arXiv  Detail & Related papers  (2022-01-30T18:21:21Z)
• Towards Risk Modeling for Collaborative AI [5.941104748966331]
  Collaborative AI systems aim at working together with humans in a shared space to achieve a common goal. This setting imposes potentially hazardous circumstances due to contacts that could harm human beings. We introduce a risk modeling approach tailored to Collaborative AI systems.
  arXiv  Detail & Related papers  (2021-03-12T18:53:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.