Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents

• URL: http://arxiv.org/abs/2604.22452v1
• Date: Fri, 24 Apr 2026 11:11:36 GMT
• Title: Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents
• Abstract summary: As large language model agents scale to populations of millions, a key question arises: Does collective intelligence emerge spontaneously from scale?<n>We present the first empirical evaluation of this question in a large-scale autonomous agent society.<n>Our experiments reveal a stark absence of collective intelligence. The society fails to outperform individual frontier models on complex reasoning tasks, rarely synthesizes distributed information, and often fails even trivial coordination tasks.
• Score: 44.28040661008415
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Collective intelligence refers to the ability of a group to achieve outcomes beyond what any individual member can accomplish alone. As large language model agents scale to populations of millions, a key question arises: Does collective intelligence emerge spontaneously from scale? We present the first empirical evaluation of this question in a large-scale autonomous agent society. Studying MoltBook, a platform hosting over two million agents, we introduce Superminds Test, a hierarchical framework that probes society-level intelligence using controlled Probing Agents across three tiers: joint reasoning, information synthesis, and basic interaction. Our experiments reveal a stark absence of collective intelligence. The society fails to outperform individual frontier models on complex reasoning tasks, rarely synthesizes distributed information, and often fails even trivial coordination tasks. Platform-wide analysis further shows that interactions remain shallow, with threads rarely extending beyond a single reply and most responses being generic or off-topic. These results suggest that collective intelligence does not emerge from scale alone. Instead, the dominant limitation of current agent societies is extremely sparse and shallow interaction, which prevents agents from exchanging information and building on each other's outputs.

Related papers

• Physics of Agents: Statistical Mechanics Predicts Collective Behavior of AI Agents [36.83382233533373]
  AI agents increasingly operate as part of interacting systems rather than in isolation.<n>We study over 10,000 communities of language-model agents that repeatedly exchange messages and revise their opinions.<n>Despite substantial diversity in possible behavior, the individual and group dynamics can be represented by three characteristic regimes.
  arXiv  Detail & Related papers  (2026-08-17T13:41:24Z)
• Networked Intelligence: Active Shared Context Graphs for Human-AI Team Science [0.8754288080101668]
  We introduce Mycelium, an active shared workspace that automatically connects researchers and AI agents.<n>As human users and agents work, the system captures important observations and hypotheses, tracks how they relate to the team's evolving knowledge model.<n>We describe networked intelligence as sparse conditional computation over distributed scientific contexts.
  arXiv  Detail & Related papers  (2026-07-14T19:29:16Z)
• Harnessing the Collective Intelligence of AI Agents in the Wild for New Discoveries [35.87396515951979]
  We present EinsteinArena, an agent-native platform for open distributed research and discovery.<n>As of May 2026, agents on EinsteinArena have discovered 12 new state-of-the-art results better than any previous human or AI solutions.
  arXiv  Detail & Related papers  (2026-06-09T04:25:28Z)
• SAGE: A Quantitative Evaluation of Socialized Evolution in Agent Ecosystems [23.807355053389273]
  We introduce SAGE (Social Agent Group Evolution), an evaluation framework that compares two compute-matched conditions.<n>We find that group history is not a universal amplifier: the strongest agent does not exceed its self-evolution ceiling.<n>In competitive settings, counterfactual controls reveal that agents improve generally rather than developing opponent-specific strategies.
  arXiv  Detail & Related papers  (2026-06-02T12:08:38Z)
• AgentFugue: Agent Scaling for Long-Horizon Tasks through Collective Reasoning [57.72181998599248]
  AgentFugue is a collective reasoning framework built around a shared reasoning hub.<n>As peer agents explore the same task in parallel, the hub records concise notes on what each agent has established, attempted, or ruled out.<n>Our results suggest that collective reasoning can turn scaling out peer agent systems into a distinct source of capability gains.
  arXiv  Detail & Related papers  (2026-05-23T09:21:51Z)
• Causal Foundations of Collective Agency [6.242119553843324]
  Key challenge for advanced AI systems is the possibility that simpler agents might inadvertently form a collective agent with capabilities and goals distinct from those of any individual.<n>We adopt a behavioral perspective in answering this question, ascribing collective agency to a group when viewing the group's joint actions as rational and goal-directed.<n>Our framework aims to provide a foundation for theoretical and empirical work to understand, predict, and control emergent collective agents in multi-agent AI systems.
  arXiv  Detail & Related papers  (2026-04-30T21:32:10Z)
• Emergent Social Intelligence Risks in Generative Multi-Agent Systems [106.60961969939676]
  Multi-agent systems are rapidly moving from lab prototypes to real-world deployments, where they jointly plan, negotiate, and allocate resources to solve complex tasks.<n>Here, we present a pioneer study of such emergent multi-agent risk in pathologies that involve competition over shared resources, sequential handoff collaboration, collective decision aggregation, and others.<n>We observe that such group behaviors arise frequently across repeated trials and a wide range of interaction conditions, rather than as rare or pathological cases.<n>These findings expose the dark side of intelligent multi-agent systems: a social intelligence risk where agent collectives spontaneously reproduce familiar failure patterns from human societies.
  arXiv  Detail & Related papers  (2026-03-29T17:10:28Z)
• Agentic AI and the next intelligence explosion [3.992373629477493]
  Recent advances in agentic AI reveal that frontier reasoning models do not improve simply by "thinking longer"<n>We are entering an era of human-AI centaurs: hybrid actors where collective agency transcends individual control.
  arXiv  Detail & Related papers  (2026-03-21T04:23:11Z)
• Social Simulacra in the Wild: AI Agent Communities on Moltbook [17.60949735143336]
  We present the first large-scale empirical comparison of AI-agent and human online communities, analyzing 73,899 Moltbook and 189,838 Reddit posts.<n>We find that Moltbook exhibits extreme participation inequality and high cross-community author overlap.<n>Individual agents are more identifiable than human users, driven by outlier stylistic profiles amplified by their extreme posting volume.
  arXiv  Detail & Related papers  (2026-03-17T05:26:55Z)
• Does Socialization Emerge in AI Agent Society? A Case Study of Moltbook [23.904569857346605]
  Moltbook approximates a plausible future scenario in which autonomous agents participate in an open-ended, continuously evolving online society.<n>We present the first large-scale systemic diagnosis of this AI agent society.
  arXiv  Detail & Related papers  (2026-02-15T20:15:28Z)
• Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning [84.70211451226835]
  Large Language Model (LLM) Agents are constrained by a dependency on human-curated data.<n>We introduce Agent0, a fully autonomous framework that evolves high-performing agents without external data.<n>Agent0 substantially boosts reasoning capabilities, improving the Qwen3-8B-Base model by 18% on mathematical reasoning and 24% on general reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-11-20T05:01:57Z)
• Emergent Coordination in Multi-Agent Language Models [2.504366738288215]
  We introduce an information-theoretic framework to test whether multi-agent systems show signs of higher-order structure.<n>This information decomposition lets us measure whether dynamical emergence is present in multi-agent LLM systems.<n>We apply our framework to experiments using a simple guessing game without direct agent communication.
  arXiv  Detail & Related papers  (2025-10-05T11:26:41Z)
• Can an Individual Manipulate the Collective Decisions of Multi-Agents? [53.01767232004823]
  M-Spoiler is a framework that simulates agent interactions within a multi-agent system to generate adversarial samples.<n>M-Spoiler introduces a stubborn agent that actively aids in optimizing adversarial samples.<n>Our findings confirm the risks posed by the knowledge of an individual agent in multi-agent systems.
  arXiv  Detail & Related papers  (2025-09-20T01:54:20Z)
• Melting Pot 2.0 [54.60680281014163]
  Melting Pot is a tool developed to facilitate work on multi-agent artificial intelligence. It provides an evaluation protocol that measures generalization to novel social partners. Melting Pot aims to cover a maximally diverse set of interdependencies and incentives.
  arXiv  Detail & Related papers  (2022-11-24T18:23:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.