When LLM Judge Scores Look Good but Best-of-N Decisions Fail

• URL: http://arxiv.org/abs/2603.12520v1
• Date: Thu, 12 Mar 2026 23:40:03 GMT
• Title: When LLM Judge Scores Look Good but Best-of-N Decisions Fail
• Abstract summary: Large language models are often used as judges to score candidate responses, then validated with a single global metric.<n>This can be misleading when the real deployment task is best-of-n selection within a prompt.<n>For judge-based selection, the relevant audit should report within-prompt signal, tie rates, and recovery/top-1 accuracy, not global agreement alone.
• Score: 0.29465623430708904
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are often used as judges to score candidate responses, then validated with a single global metric such as correlation with reference labels. This can be misleading when the real deployment task is best-of-n selection within a prompt. In a 5,000-prompt best-of-4 benchmark from Chatbot Arena, a judge with moderate global correlation (r = 0.47) captures only 21.0% of the improvement that perfect selection would achieve over random choice. The gap arises because global agreement is driven largely by prompt-level baseline effects, while selection depends on within-prompt ranking: within-prompt correlation is only r_within = 0.27, and coarse pointwise scoring creates ties in 67% of pairwise comparisons. In a matched-pair best-of-2 audit, explicit pairwise judging recovers much of this lost signal, raising recovery from 21.1% to 61.2%. For judge-based selection, the relevant audit should report within-prompt signal, tie rates, and recovery/top-1 accuracy, not global agreement alone.

Related papers

• Small Language Models as Judges for Rubric-Based Reinforcement Learning [57.707881387886516]
  We study whether smaller language models can serve as efficient and reliable rubric-based judges.<n>We compare three ways of extracting criterion-level judgments from small models: Generative verdicts, Yes/No Logprob margins, and Probe judges.<n>Across both datasets, the Qwen3-1.7B Probe judge achieves the strongest criterion-level agreement among these methods.
  arXiv  Detail & Related papers  (2026-08-30T20:00:17Z)
• Beyond Accuracy: A Dual-Judge Evaluation Protocol for Vision-Language Models in Legally Grounded Tasks [1.2555090617748867]
  We contribute a dual-judge protocol that pairs a standard 0-10 quality judge with a strict binary semantic-equivalence judge against a human-curated reference.<n>We study a controlled, visually grounded regulatory task, whose meaning is a codified question with a known reference for every input.
  arXiv  Detail & Related papers  (2026-08-25T08:49:46Z)
• Counterfactual, Per-Decision Bias Auditing for Automated Hiring: Localizing and Explaining Disparate Impact in Applicant Tracking Systems [0.0]
  We present the AI Bias Firewall (AIBF), a method that audits an applicant tracking system one decision at a time.<n>AIBF neutralizes a candidate's protected-attribute proxies, re-scores the decision, and measures the resulting counterfactual shift.<n>We evaluate on two real public datasets, Adult and COMPAS, rather than on synthetic data.
  arXiv  Detail & Related papers  (2026-08-21T18:19:20Z)
• When Many Answers Are Valid, Voting Fails: Symbolic Verification for Best-of-K Causal Reasoning in LLMs [42.32694162421078]
  We introduce CALVER, a training-free symbolic verifier that scores structured traces against Pearl's causal criteria.<n>On CLEAR find-one-valid queries that admit multiple graph-valid answers, CALVER reaches 42.1% where plurality, a reward model, an LLM judge, and model confidence remain near 30%.
  arXiv  Detail & Related papers  (2026-08-04T11:45:46Z)
• PACE: A Proxy for Agentic Capability Evaluation [60.3743414796937]
  We introduce PACE, a framework that constructs proxy benchmarks by selecting instances from existing non-agentic evaluations.<n>Experiments across 14 models, 4 agentic benchmarks, and 19 non-agentic benchmarks show that PACE-Bench predicts agentic scores with leave-one-out cross-validation (LOOCV) mean absolute error (MAE) under 4%, Spearman correlation above 0.80, and pairwise model-ranking accuracy around 85%, all at much less than 1% of the full agentic evaluation cost.
  arXiv  Detail & Related papers  (2026-07-02T10:59:03Z)
• The Coin Flip Judge? Reliability and Bias in LLM-as-a-Judge Evaluation [0.0]
  We study identical evaluations on 29 tasks spanning 10 categories using two OpenAI judge models.<n>Across judges, pairwise preferences flip on average 13.6% of the time, with 28% of questions exceeding a 20% flip rate and one question reaching 56%.<n>Cross-judge agreement is only 76%, semantically equivalent prompt templates change majority outcomes in 25% of tested cases, and deterministic decoding reduces but does not eliminate inconsistency.
  arXiv  Detail & Related papers  (2026-04-23T18:19:10Z)
• Putting HUMANS first: Efficient LAM Evaluation with Human Preference Alignment [53.72927532626824]
  We show that subsets of just 50 examples (0.3% of data) can achieve over 0.93 Pearson correlation with full benchmark scores.<n>To better predict preferences, we trained regression models on these selected subsets, achieving 0.98 correlation.<n>This demonstrates that in regression modeling, well-curated subsets outpredict the full benchmark, showing quality over quantity.
  arXiv  Detail & Related papers  (2026-04-20T00:57:31Z)
• Re-Rankers as Relevance Judges [65.37611299805856]
  We reproduce re-rankers in a re-ranker-as-relevance-judge setup.<n>We perform experiments on TREC-DL 2019 to 2023 with 8 re-rankers from 3 families, ranging from 220M to 32B, and analyse the evaluation bias exhibited by re-ranker-based judges.
  arXiv  Detail & Related papers  (2026-01-08T00:02:59Z)
• ORCA: Open-ended Response Correctness Assessment for Audio Question Answering [41.72231074041232]
  We present ORCA, a framework that models the variability in human judgments using Beta distributions to predict both expected correctness and uncertainty.<n>We collected 11,721 annotations across 3,580 question-answer pairs from 15 LALMs on two audio QA benchmarks, achieving inter-annotator agreement of 0.82 (Krippendorff's alpha)
  arXiv  Detail & Related papers  (2025-11-28T14:41:48Z)
• Fortytwo: Swarm Inference with Peer-Ranked Consensus [36.94429692322632]
  We present Fortytwo, a novel protocol that leverages swarm intelligence principles and distributed pairwise ranking consensus to achieve superior performance in AI inference.<n>Using pairwise ranking with a custom Bradley-Terry-style aggregation model, we demonstrate that swarm inference substantially outperforms majority voting.
  arXiv  Detail & Related papers  (2025-10-27T23:19:48Z)
• Reference-Free Rating of LLM Responses via Latent Information [53.463883683503106]
  We study the common practice of asking a judge model to assign Likert-scale scores to free-text responses.<n>We then propose and evaluate Latent Judges, which derive scalar ratings from internal model signals.<n>Across a broad suite of pairwise and single-rating benchmarks, latent methods match or surpass standard prompting.
  arXiv  Detail & Related papers  (2025-09-29T12:15:52Z)
• TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them [58.04324690859212]
  Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks.<n>We identify two fundamental types of inconsistencies: Score-Comparison Inconsistency and Pairwise Transitivity Inconsistency.<n>We propose TrustJudge, a probabilistic framework that addresses these limitations through two key innovations.
  arXiv  Detail & Related papers  (2025-09-25T13:04:29Z)
• ObjexMT: Objective Extraction and Metacognitive Calibration for LLM-as-a-Judge under Multi-Turn Jailbreaks [12.396822247035578]
  We present exMT, a benchmark for objective extraction and metacognition.<n>Given a multi-turn transcript, a model must output a one-sentence base objective and a self-reported confidence.<n> Accuracy is scored by similarity to gold objectives, then thresholded once on 300 calibration items.
  arXiv  Detail & Related papers  (2025-08-23T03:32:04Z)
• Confidence and Stability of Global and Pairwise Scores in NLP Evaluation [7.094351095888013]
  This paper empirically investigates the strengths and weaknesses of both global scores and pairwise comparisons.<n>We found that while global scores provide more reliable overall rankings, they can underestimate strong models with rare, significant errors or low confidence.
  arXiv  Detail & Related papers  (2025-07-02T12:05:22Z)
• Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios [49.53589774730807]
  Multimodal large language models (MLLMs) have recently achieved state-of-the-art performance on tasks ranging from visual question answering to video understanding.<n>We reveal a response uncertainty phenomenon: twelve state-of-the-art open-source MLLMs overturn a previously correct answer in 65% of cases after receiving a single deceptive cue.
  arXiv  Detail & Related papers  (2024-11-05T01:11:28Z)
• Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics [64.81682222169113]
  How reliably an automatic summarization evaluation metric replicates human judgments of summary quality is quantified by system-level correlations. We identify two ways in which the definition of the system-level correlation is inconsistent with how metrics are used to evaluate systems in practice.
  arXiv  Detail & Related papers  (2022-04-21T15:52:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.