More Debate, Same Evidence: Structural Limits of Homogeneous Multi-Agent Groundedness

• URL: http://arxiv.org/abs/2608.00243v1
• Date: Fri, 31 Jul 2026 19:40:15 GMT
• Title: More Debate, Same Evidence: Structural Limits of Homogeneous Multi-Agent Groundedness
• Abstract summary: We evaluate a homogeneous three-agent panel on six public fact-verification benchmarks.<n>Relative to a fixed single-agent reference, the panel's system-level accuracy difference ranges from $+8.5$ to $-4.4$ percentage points.
• Score: 1.9036571490366498
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM) judges are increasingly organized as multi-agent panels under the assumption that exchanging critiques improves judgment quality. We test this assumption for \emph{groundedness verification}, where a judge must determine whether a claim is supported by the supplied evidence. We evaluate a homogeneous three-agent panel on six public fact-verification and hallucination-detection benchmarks. Relative to a fixed single-agent reference, the panel's system-level accuracy difference ranges from $+8.5$ to $-4.4$ percentage points: two datasets show reliable gains, one shows a reliable loss, and three are statistically inconclusive. Because the reference and panel use different model variants, these differences characterize the complete systems rather than isolate a causal debate effect.

Related papers

• NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment [54.4265627247104]
  Large language models (LLMs) are increasingly used in peer review at major AI conferences.<n>Existing benchmarks assess novelty as a single holistic score.<n>We present NovGauge, a human-anchored benchmark for fine-grained novelty assessment diagnosis.
  arXiv  Detail & Related papers  (2026-09-10T08:34:56Z)
• Forged Peer Judgments Mislead Multimodal LLM Judge Panels: Source-Blind Anchoring and Panel-Consensus Verification [9.710464466895521]
  Quoting independent visual judgments creates large anchoring gaps under both self and peer framing.<n>We introduce panel-consensus verification, which cross-checks a quote against independently collected blind votes.
  arXiv  Detail & Related papers  (2026-08-08T04:55:45Z)
• Diverse Evidence, Better Forecasts: Multi-Agent Deliberation Under Information Asymmetry [1.8983854039418835]
  We show that when all agents are given identical evidence, deliberation collapses into herding rather than genuine belief revision.<n>We propose a framework combining relevance-aware evidence routing, rationale-based iterative deliberation, and confidence-weighted aggregation.<n>On PolyGym, a benchmark of 375 binary forecasting questions, InfoDelphi outperforms the strongest single-agent and multi-agent baselines.
  arXiv  Detail & Related papers  (2026-07-02T03:41:25Z)
• PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception [75.70273182182397]
  PerceptionRubrics is a rubric-based evaluation framework.<n>It addresses the gap between saturated benchmark scores and real-world brittleness.<n>It pairs 1,038 information-dense images with over 10,000 instance-specific rubrics.
  arXiv  Detail & Related papers  (2026-06-26T17:59:15Z)
• AgentFairBench: Do LLM Agents Discriminate When They Act? [2.3004655342211078]
  AgentFairBench is a cheap, reproducible, multi-domain benchmark for demographic disparity in the actions of LLM agents.<n>It spans three regulator-anchored domains: hiring, lending, and medical triage.<n>A NumPy-only harness computes counterfactual flip rate, mean absolute score difference (MASD), action-rate disparity, and tool-invocation disparity.
  arXiv  Detail & Related papers  (2026-06-15T13:50:26Z)
• Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels [0.0]
  LLM-as-a-judge panels aggregate votes from multiple models.<n>We quantify how far their reliability falls short of the independent-voting ideal.
  arXiv  Detail & Related papers  (2026-05-28T11:48:17Z)
• LLMs Show No Signs Of Individuated Metacognition [0.023227405857540805]
  We decompose binary confidence judgements from 20 Large Language Models.<n>We ask whether two models that differ in confidence also differ in performance.<n>We find no evidence for significant verbalised individuated metacognition in any tested domain.
  arXiv  Detail & Related papers  (2026-05-22T23:54:33Z)
• Knowing When Not to Answer: Evaluating Abstention in Multimodal Reasoning Systems [52.83669998269706]
  Abstention has been studied in text-only settings but remains underexplored multimodally.<n>Current benchmarks either ignore unanswerability or rely on coarse methods that miss realistic failure modes.<n>We introduce MM-AQA, a benchmark that constructs unanswerable instances from answerable ones via transformations along two axes.
  arXiv  Detail & Related papers  (2026-04-16T09:23:22Z)
• Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents [66.97968363332465]
  We introduce Claw-Eval, an end-to-end evaluation suite addressing all three gaps in agent benchmarks.<n>Claw-Eval comprises 300 human-verified tasks spanning 9 categories across three groups.<n>Every agent action is recorded through three independent evidence channels.
  arXiv  Detail & Related papers  (2026-04-07T17:43:18Z)
• C2-Faith: Benchmarking LLM Judges for Causal and Coverage Faithfulness in Chain-of-Thought Reasoning [0.6138671548064355]
  Large language models (LLMs) are increasingly used as judges of chain-of-thought (CoT) reasoning.<n>We introduce C2-Faith, a benchmark that targets two complementary dimensions of faithfulness: causality (does each step logically follow from prior context?) and coverage.<n>We evaluate three frontier judges under three tasks: binary causal detection, causal step localization, and coverage scoring.
  arXiv  Detail & Related papers  (2026-03-05T13:36:47Z)
• A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness [57.510025257780306]
  We show that existing validation protocols fail to account for substantial distribution shifts inherent to red-teaming.<n>We propose ReliableBench, a benchmark of behaviors that remain more consistently judgeable, and JudgeStressTest, a dataset designed to expose judge failures.
  arXiv  Detail & Related papers  (2026-02-04T15:13:35Z)
• All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detection [67.89888669159899]
  RFC Bench is a benchmark for evaluating large language models on financial misinformation under realistic news.<n>The benchmark defines two complementary tasks: reference free misinformation detection and comparison based diagnosis.
  arXiv  Detail & Related papers  (2026-01-07T18:18:28Z)
• RoboView-Bias: Benchmarking Visual Bias in Embodied Agents for Robotic Manipulation [67.38036090822982]
  We propose RoboView-Bias, the first benchmark specifically designed to quantify visual bias in robotic manipulation.<n>We create 2,127 task instances that enable robust measurement of biases induced by individual visual factors and their interactions.<n>Our results highlight that systematic analysis of visual bias is a prerequisite for developing safe and reliable general-purpose embodied agents.
  arXiv  Detail & Related papers  (2025-09-26T13:53:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.