Representational Collapse in Multi-Agent LLM Committees: Measurement and Diversity-Aware Consensus

• URL: http://arxiv.org/abs/2604.03809v1
• Date: Sat, 04 Apr 2026 17:30:23 GMT
• Title: Representational Collapse in Multi-Agent LLM Committees: Measurement and Diversity-Aware Consensus
• Abstract summary: Multi-agent LLM committees replicate the same model under different role prompts and aggregate outputs by majority vote.<n>We embed each agent's chain-of-thought rationale and measure pairwise similarity across 100 GSM8K questions with three Qwen2.5-14B agents.<n>DALC, a training-free consensus protocol that computes diversity weights from embedding geometry, reaches 87% on GSM8K versus 84% for self-consistency at 26% lower token cost.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-agent LLM committees replicate the same model under different role prompts and aggregate outputs by majority vote, implicitly assuming that agents contribute complementary evidence. We embed each agent's chain-of-thought rationale and measure pairwise similarity: across 100 GSM8K questions with three Qwen2.5-14B agents, mean cosine similarity is 0.888 and effective rank is 2.17 out of 3.0, a failure mode we term representational collapse. DALC, a training-free consensus protocol that computes diversity weights from embedding geometry, reaches 87% on GSM8K versus 84% for self-consistency at 26% lower token cost. Ablation experiments reveal 1-3 point per-protocol run-to-run variance, confirm that hint sharing contributes more than diversity weighting alone, and show that encoder choice strongly modulates collapse severity (cosine 0.908 with mxbai versus 0.888 with nomic) and downstream accuracy. The more robust finding is that collapse is measurable, worsens on harder tasks, and that the choice of embedding proxy is a first-order design decision for any latent communication protocol.

Related papers

• Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate [5.650336594658653]
  We show that a conventional answer flip rate conflates three mechanisms: spontaneous instability, stance-induced conformity, and reasoning-induced persuasion.<n>Our three-source decomposition framework isolates each through controlled counterfactual conditions.
  arXiv  Detail & Related papers  (2026-05-30T17:41:11Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• The Cost of Consensus: Isolated Self-Correction Prevails Over Unguided Homogeneous Multi-Agent Debate [5.51170856062205]
  We study teams of homogeneous agents across $R=3$ debate rounds on two high-difficulty benchmarks.<n>We decompose debate failure into three model-dependent pathways: sycophantic conformity, peer rationales, and consensus collapse.<n>Our results indicate that, within the 7-8B parameter class, homogeneous teams without structured roles do not benefit from unguided peer exchange.
  arXiv  Detail & Related papers  (2026-04-29T14:33:57Z)
• Preserving Disagreement: Architectural Heterogeneity and Coherence Validation in Multi-Agent Policy Simulation [0.0]
  Multi-agent deliberation systems using large language models (LLMs) are increasingly proposed for policy simulation.<n>They suffer from artificial consensus: evaluator agents converge on the same option regardless of their assigned value perspectives.<n>We present the AI Council, a three-phase deliberation framework, and conduct 120 deliberations across two policy scenarios to test two interventions.
  arXiv  Detail & Related papers  (2026-04-29T11:47:28Z)
• PROXIMA: A Reliability Scoring Framework for Proxy Metrics in Online Controlled Experiments [0.0]
  PROXIMA scores proxy reliability through a composite of three complementary dimensions.<n>It directly audits whether a candidate proxy leads to correct launch decisions.<n>Fragility analysis reveals that recommendation domains exhibit substantially higher segment-level fragility.
  arXiv  Detail & Related papers  (2026-04-15T19:10:53Z)
• Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents [66.97968363332465]
  We introduce Claw-Eval, an end-to-end evaluation suite addressing all three gaps in agent benchmarks.<n>Claw-Eval comprises 300 human-verified tasks spanning 9 categories across three groups.<n>Every agent action is recorded through three independent evidence channels.
  arXiv  Detail & Related papers  (2026-04-07T17:43:18Z)
• The Specification Gap: Coordination Failure Under Partial Knowledge in Code Agents [0.0]
  Two-agent integration accuracy drops from 58% to 25% as detail is removed.<n>A factorial recovery experiment shows that restoring the full specification alone recovers the single-agent ceiling.<n>The gap is not merely a consequence of hidden information, but reflects the difficulty of producing compatible code without shared decisions.
  arXiv  Detail & Related papers  (2026-03-25T13:18:26Z)
• Semantic Invariance in Agentic AI [2.7821684674538347]
  Large Language Models increasingly serve as autonomous reasoning agents in decision support, scientific problem-solving, and multi-agent coordination systems.<n> deploying LLM agents in consequential applications requires assurance that their reasoning remains stable under semantically equivalent input variations.<n>Standard benchmark evaluations, which assess accuracy on fixed, canonical problem formulations, fail to capture this critical reliability dimension.
  arXiv  Detail & Related papers  (2026-03-13T17:08:44Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Internalizing Self-Consistency in Language Models: Multi-Agent Consensus Alignment [22.305033366660187]
  Language Models (LMs) are inconsistent reasoners, often generating contradictory responses to identical prompts.<n>We formalize self-consistency as an intrinsic property of well-aligned reasoning models and introduce Multi-Agent Consensus Alignment (MACA)<n>MACA enables agents to teach themselves to be more decisive and concise, and better leverage peer insights in multi-agent settings without external supervision.
  arXiv  Detail & Related papers  (2025-09-18T17:27:28Z)
• ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs [61.07130026622437]
  Large Language Models (LLMs) still struggle with natural language reasoning tasks. Motivated by the society of minds, we propose ReConcile. A multi-model multi-agent framework designed as a round table conference among diverse LLM agents.
  arXiv  Detail & Related papers  (2023-09-22T17:12:45Z)
• On the Complexity of Multi-Agent Decision Making: From Learning in Games to Partial Monitoring [105.13668993076801]
  A central problem in the theory of multi-agent reinforcement learning (MARL) is to understand what structural conditions and algorithmic principles lead to sample-efficient learning guarantees. We study this question in a general framework for interactive decision making with multiple agents. We show that characterizing the statistical complexity for multi-agent decision making is equivalent to characterizing the statistical complexity of single-agent decision making.
  arXiv  Detail & Related papers  (2023-05-01T06:46:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.