When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines

• URL: http://arxiv.org/abs/2603.20324v1
• Date: Fri, 20 Mar 2026 00:50:53 GMT
• Title: When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines
• Abstract summary: Multi-agent LLM pipelines produce contradictory evidence on whether team diversity improves output quality.<n>We propose a resolution by identifying the selection bottleneck that determines whether diversity helps or hurts.<n>Our results suggest that selector quality may be a more impactful design lever than generator diversity in single-round generate-then-select pipelines.
• Score: 0.0
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Multi-agent LLM pipelines produce contradictory evidence on whether team diversity improves output quality: heterogeneous Mixture-of-Agents teams outperform single models, yet homogeneous Self-MoA teams consistently win under synthesis-based aggregation. We propose a resolution by identifying the selection bottleneck -- a crossover threshold in aggregation quality that determines whether diversity helps or hurts. Under this model, we obtain a closed-form crossover threshold $s^*$ (Proposition 1) that separates the regimes where diversity helps and hurts. In a targeted experiment spanning 42 tasks across 7 categories ($N=210$), a diverse team with judge-based selection achieves a win rate of 0.810 against a single-model baseline, while a homogeneous team scores 0.512 -- near chance (Glass's $Δ= 2.07$). Judge-based selection outperforms MoA-style synthesis by $Δ_{\mathrm{WR}} = +0.631$ -- the synthesis approach is preferred over the baseline in zero of 42 tasks by the judge panel. A decoupled evaluation with independent judges confirms all directional findings (Spearman $ρ= 0.90$). Exploratory evidence suggests that including a weaker model improves performance while reducing cost ($p < 10^{-4}$, not pre-registered). Our results suggest that selector quality may be a more impactful design lever than generator diversity in single-round generate-then-select pipelines.

Related papers

• Spokes: Optimizing for Diverse Pretraining Data Selection [82.66872118512403]
  We introduce a probabilistic diversification framework based on the G-Vendi score, optimized via exponentiated gradient descent.<n>Our method produces subsets substantially more diverse than those obtained via random sampling.<n>We evaluate our approach on FineWeb and DCLM, where it consistently outperforms existing methods.
  arXiv  Detail & Related papers  (2026-06-13T09:28:46Z)
• FLOWREADER: Min-Cost Flow Optimization for Multi-Modal Long Document Q&A [6.071121358322323]
  We introduce FLOWREADER, which reframes evidence assembly as a min-cost flow problem on a multimodal node graph.<n>The optimal flow is decomposed into candidate evidence paths, a compact non-redundant subset is selected by entropy-regularized replicator dynamics.<n>FLOWREADER is best on the two subsets dominated by fragmented evidence PaperTab and SlideVQA.
  arXiv  Detail & Related papers  (2026-06-05T13:05:00Z)
• Instance-Optimal Estimation with Multiple LLM Judges on a Budget [84.31744861038106]
  We formalize this question as *budgeted heteroskedastic multi-judge estimation*.<n>Given $K$ prompt-response pairs, $J$ judges with known costs, and unknown query-judge variances, the goal is to estimate a bounded score vector while minimizing an $ell_p$-error.<n>We prove that EST-IVWE matches the oracle IVWE rate up to lower-order terms in the budget.
  arXiv  Detail & Related papers  (2026-05-22T08:26:08Z)
• Correcting Selection Bias in Sparse User Feedback for Large Language Model Quality Estimation: A Multi-Agent Hierarchical Bayesian Approach [0.9558392439655014]
  We propose a three-agent hierarchical Bayesian pipeline that does not require ground-truth labels on individual interactions.<n>A mild prior on the feedback channel (typical positive-feedback rate and negative-to-positive ratio) keeps Hierarchical-Informed within 4-13 pp of $Qstar$ as the bias ratio sweeps.<n>Without channel-side priors, every weak-prior variant misses $Qstar$ by 22-33 pp.
  arXiv  Detail & Related papers  (2026-05-12T14:22:06Z)
• Attributing Emergence in Million-Agent Systems [68.53670424791751]
  Large language models (LLMs) can simulate human-like reasoning and decision-making in individual agents.<n>Such studies require attributing macro emergence to individual agents.<n>We adapt Aumann--Shapley path-integral attribution to LLM-powered MAS at million-agent scale.
  arXiv  Detail & Related papers  (2026-05-12T01:49:41Z)
• BiAxisAudit: A Novel Framework to Evaluate LLM Bias Across Prompt Sensitivity and Response-Layer Divergence [22.315546054051143]
  Bias audits of large language models now operate within governance frameworks such as the EU AI Act.<n>We introduce textscBiAxisAudit, a protocol that reports each bias score together with a reliability estimate on two axes.
  arXiv  Detail & Related papers  (2026-05-09T16:26:49Z)
• Budget-Sensitive Discovery Scoring: A Formally Verified Framework for Evaluating AI-Guided Scientific Selection [0.0]
  Budget-Sensitive Discovery Score (BSDS) penalizes false discoveries at each budget level.<n>Discovery Quality Score (DQS) provides a single summary statistic that no proposer can inflate by performing well at a cherry-picked budget.<n> framework applies to any setting where candidates are selected under budget constraints and asymmetric error costs.
  arXiv  Detail & Related papers  (2026-03-12T18:09:53Z)
• Chaotic Dynamics in Multi-LLM Deliberation [0.0]
  We quantify inter-run sensitivity using an empirical Lyapunov exponent ($hat$) derived from trajectory divergence in committee mean preferences.<n>We identify two independent routes to instability: role differentiation in homogeneous committees and model heterogeneity in no-role committees.<n>These results support stability auditing as a core design requirement for multi-LLM governance systems.
  arXiv  Detail & Related papers  (2026-03-10T02:59:11Z)
• $V_1$: Unifying Generation and Self-Verification for Parallel Reasoners [69.66089681814013]
  $V_$ is a framework that unifies generation and verification through efficient pairwise ranking.<n>$V_$-Infer improves Pass@1 by up to $10%$ over pointwise verification.<n>$V_$-PairRL achieves $7$--$9%$ test-time scaling gains over standard RL and pointwise joint training.
  arXiv  Detail & Related papers  (2026-03-04T17:22:16Z)
• RouteMoA: Dynamic Routing without Pre-Inference Boosts Efficient Mixture-of-Agents [91.0187958746262]
  RouteMoA is an efficient mixture-of-agents framework with dynamic routing.<n>It employs a lightweight scorer to perform initial screening by predicting coarse-grained performance from the query.<n>It refines these scores through lightweight self- and cross-assessment based on existing model outputs, providing posterior correction without additional inference.
  arXiv  Detail & Related papers  (2026-01-26T04:22:22Z)
• Balancing Centralized Learning and Distributed Self-Organization: A Hybrid Model for Embodied Morphogenesis [0.0]
  We investigate how to couple a learnable brain-like'' controller to a cell-like'' Gray--Scott substrate to steer pattern formation with minimal effort.<n>A compact convolutional policy is embedded in a differentiable PyTorch reaction--diffusion simulator.
  arXiv  Detail & Related papers  (2025-11-13T09:05:27Z)
• DISCO: Diversifying Sample Condensation for Efficient Model Evaluation [59.01400190971061]
  Costly evaluation reduces inclusivity, slows the cycle of innovation, and worsens environmental impact.<n>We argue that promoting diversity among samples is not essential; what matters is to select samples thatmaximise diversity in model responses.<n>Our method, $textbfDiversifying Sample Condensation (DISCO)$, selects the top-k samples with the greatest model disagreements.
  arXiv  Detail & Related papers  (2025-10-09T08:53:59Z)
• Creativity Benchmark: A benchmark for marketing creativity for large language models [0.509780930114934]
  Creativity Benchmark is an evaluation framework for large language models (LLMs) in marketing creativity.<n>The benchmark covers 100 brands (12 categories) and three prompt types (Insights, Ideas, Wild Ideas)
  arXiv  Detail & Related papers  (2025-09-05T04:44:29Z)
• Confidence-Based Model Selection: When to Take Shortcuts for Subpopulation Shifts [119.22672589020394]
  We propose COnfidence-baSed MOdel Selection (CosMoS), where model confidence can effectively guide model selection. We evaluate CosMoS on four datasets with spurious correlations, each with multiple test sets with varying levels of data distribution shift.
  arXiv  Detail & Related papers  (2023-06-19T18:48:15Z)
• Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexity [67.02490430380415]
  We show that model-based MARL achieves a sample complexity of $tilde O(|S||B|(gamma)-3epsilon-2)$ for finding the Nash equilibrium (NE) value up to some $epsilon$ error. We also show that such a sample bound is minimax-optimal (up to logarithmic factors) if the algorithm is reward-agnostic, where the algorithm queries state transition samples without reward knowledge.
  arXiv  Detail & Related papers  (2020-07-15T03:25:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.