When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals

• URL: http://arxiv.org/abs/2607.08065v1
• Date: Thu, 09 Jul 2026 02:46:51 GMT
• Title: When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals
• Abstract summary: LLM-as-judge is increasingly the default for evaluating AI systems in enterprise pipelines.<n>We show that consistency -- agreement among judges, or a model's own samples -- indicates correctness.<n>We ask when agreement is nonetheless a usable proxy, in a large-scale cross-runner study.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM-as-judge (Zheng et al., 2023) is increasingly the default for evaluating AI systems in enterprise pipelines, often scaled to ensembles (Verga et al., 2024) or "mixture-of-experts" (Shazeer et al., 2017) panels of judges. These systems share a key assumption: that consistency -- agreement among judges, or among a model's own samples -- indicates correctness. We show this assumption is unreliable. Agreement is not accuracy: a model can agree with itself, and different models can agree with each other, out of shared bias, a memorized heuristic, or an option-position prior rather than truth. We ask when agreement is nonetheless a usable proxy, in a large-scale cross-runner study: 53 runners drew K=50 samples for assigned overlapping cases across comparisons of model tier, prompting, and scale on GPQA Diamond and AIME -- 265,000 samples. Using majority-correctness as the deployment label and a hierarchical runner-clustered bootstrap, agreement is a positive but weak predictor (rho 0.20-0.59, all positive under item-clustered resampling) whose usefulness is regime-dependent: best for unsaturated mid-tier models and for allocating compute, and worst -- over-confident yet no more accurate -- for the most consistent frontier model (agreement >=0.8 on 77% of GPQA case-result entries, 48% of those wrong). An exploratory cross-family check on three Claude tiers shows the same frontier over-confidence, with confident errors recurring across providers above a marginal-preserving null. Self-consistency is thus a conditional proxy for correctness, not a standalone confidence score. We publicly release the de-identified per-run rows and answer distributions.

Related papers

• Can We Trust LLM Judges: A Study of Capability-Dependent Biases and Multi-Judge Ensemble for Bias Calibration [0.5294698352039444]
  We study absolute scoring tasks, which mirror more realistic use cases.<n>We show that a model's task accuracy strongly predicts its judging accuracy.<n>More capable examinee models consistently receive more lenient judgments from all judges.
  arXiv  Detail & Related papers  (2026-09-10T00:25:39Z)
• Cross-Model Agreement as a Deployment-Time Reliability Signal for Automatic Polyp Segmentation [0.639901418243611]
  In real-time colonoscopy, ground-truth annotations are unavailable at inference, so polyp segmentation models can fail silently.<n>We propose Referee-Based Quality Estimation (RBQE), a reference-free framework measuring agreement between a primary segmentation model and an independently trained referee on the same image.
  arXiv  Detail & Related papers  (2026-09-09T17:32:43Z)
• LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning [6.241883798820154]
  We study cross-model consensus, the degree to which independently trained models, each solving the problem once, agree on a final answer.<n>We treat the panel as an LLM-jury, in which the structure of agreement, not any model's score of another, is the verification signal.<n>We make this precise with a parameter-free law, derived in closed form, that predicts consensus accuracy from three measured panel statistics to a mean absolute error of $0.03$.
  arXiv  Detail & Related papers  (2026-07-11T05:57:54Z)
• CALIBER: Calibrating Confidence Before and After Reasoning in Language Models [16.376895132982938]
  We introduce CALIBER (Calibration Before and After Reasoning), which elicits both estimates each with the target matched to its information state.<n>Under this unified protocol, CALIBER reduces Expected Error (ECE) by 52.5% over the strongest single-confidence baseline on BigMathDigits.<n>On a larger 30B model, CALIBER achieves the best ECE on BigMathDigits while remaining competitive in Brier score and AUROC.
  arXiv  Detail & Related papers  (2026-06-23T08:03:20Z)
• Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Bias [0.0]
  Judge validation in practice relies on exact-match agreement, a metric that does not correct for chance and systematically overstates discriminative ability.<n>We present the largest systematic evaluation of LLM-as-a-Judge to date: 21 judges from nine providers across MT-Bench, JudgeBench, and RewardBench, evaluated under three protocols (agreement, consistency, bias audit) over 118 runs and approximately 541,000 individual judgments.
  arXiv  Detail & Related papers  (2026-06-17T19:37:13Z)
• Design and Evaluation of Multi-Agent AI Oracle Systems for Prediction Market Resolution [0.0]
  Prediction markets aggregate collective intelligence to forecast uncertain events.<n>Existing oracle systems tradeoff fast but brittle automation against accurate but costly human arbitration.<n>We evaluate whether multi-agent LLM architectures can improve oracle resolution accuracy over single-model baselines.
  arXiv  Detail & Related papers  (2026-05-29T03:44:19Z)
• Confidence-Aware Alignment Makes Reasoning LLMs More Reliable [65.44962502963378]
  We introduce CASPO, a framework that aligns token-level confidence with step-wise logical correctness without training a separate reward model.<n>During inference, we propose Confidence-aware Thought (CaT), which dynamically prunes uncertain reasoning branches with negligible O(V) latency.<n>Experiments across ten benchmarks and multiple model families show that CASPO consistently improves reasoning reliability and inference efficiency.
  arXiv  Detail & Related papers  (2026-05-08T07:08:25Z)
• Dependence-Aware Label Aggregation for LLM-as-a-Judge via Ising Models [55.94503936470247]
  Large-scale AI evaluation increasingly relies on aggregating binary judgments from $K$ annotators, including judges.<n>Most classical methods assume annotators are conditionally independent given the true label $Yin0,1$, an assumption often violated by LLM judges.<n>We study label aggregation through a hierarchy of dependence-aware models based on Ising graphical models and latent factors.
  arXiv  Detail & Related papers  (2026-01-29T21:26:50Z)
• Judging with Confidence: Calibrating Autoraters to Preference Distributions [56.17041629492863]
  We argue that a reliable autorater must learn to model the full distribution of preferences defined by a target population.<n>We present two learning methods tailored to different data conditions.<n>Our results show that finetuning autoraters with a distribution-matching objective leads to verbalized probability predictions that are better aligned with the target preference distribution.
  arXiv  Detail & Related papers  (2025-09-30T20:36:41Z)
• Reference-Free Rating of LLM Responses via Latent Information [53.463883683503106]
  We study the common practice of asking a judge model to assign Likert-scale scores to free-text responses.<n>We then propose and evaluate Latent Judges, which derive scalar ratings from internal model signals.<n>Across a broad suite of pairwise and single-rating benchmarks, latent methods match or surpass standard prompting.
  arXiv  Detail & Related papers  (2025-09-29T12:15:52Z)
• TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them [58.04324690859212]
  Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks.<n>We identify two fundamental types of inconsistencies: Score-Comparison Inconsistency and Pairwise Transitivity Inconsistency.<n>We propose TrustJudge, a probabilistic framework that addresses these limitations through two key innovations.
  arXiv  Detail & Related papers  (2025-09-25T13:04:29Z)
• When Does Confidence-Based Cascade Deferral Suffice? [69.28314307469381]
  Cascades are a classical strategy to enable inference cost to vary adaptively across samples. A deferral rule determines whether to invoke the next classifier in the sequence, or to terminate prediction. Despite being oblivious to the structure of the cascade, confidence-based deferral often works remarkably well in practice.
  arXiv  Detail & Related papers  (2023-07-06T04:13:57Z)
• Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classification [31.392067805022414]
  Variance in predictions across different trained models is a significant, under-explored source of error in fair binary classification. In practice, the variance on some data examples is so large that decisions can be effectively arbitrary. We develop an ensembling algorithm that abstains from classification when a prediction would be arbitrary.
  arXiv  Detail & Related papers  (2023-01-27T06:52:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.