Judging Against the Reference: Uncovering Knowledge-Driven Failures in LLM-Judges on QA Evaluation

• URL: http://arxiv.org/abs/2601.07506v1
• Date: Mon, 12 Jan 2026 13:05:13 GMT
• Title: Judging Against the Reference: Uncovering Knowledge-Driven Failures in LLM-Judges on QA Evaluation
• Abstract summary: Large language models (LLMs) are increasingly used as automatic judges for question answering (QA)<n>We show that when the provided reference conflicts with the judge model's parametric knowledge, the resulting scores become unreliable, substantially degrading evaluation fidelity.<n>We empirically show that this vulnerability is driven by judges' over-reliance on parametric knowledge, leading judges to disregard the given reference under conflict.
• Score: 21.864019348357303
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While large language models (LLMs) are increasingly used as automatic judges for question answering (QA) and other reference-conditioned evaluation tasks, little is known about their ability to adhere to a provided reference. We identify a critical failure mode of such reference-based LLM QA evaluation: when the provided reference conflicts with the judge model's parametric knowledge, the resulting scores become unreliable, substantially degrading evaluation fidelity. To study this phenomenon systematically, we introduce a controlled swapped-reference QA framework that induces reference-belief conflicts. Specifically, we replace the reference answer with an incorrect entity and construct diverse pairings of original and swapped references with correspondingly aligned candidate answers. Surprisingly, grading reliability drops sharply under swapped references across a broad set of judge models. We empirically show that this vulnerability is driven by judges' over-reliance on parametric knowledge, leading judges to disregard the given reference under conflict. Finally, we find that this failure persists under common prompt-based mitigation strategies, highlighting a fundamental limitation of LLM-as-a-judge evaluation and motivating reference-based protocols that enforce stronger adherence to the provided reference.

Related papers

• Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guarantees [16.73004625486562]
  Using LLMs as judges has become standard practice for evaluating model outputs at scale.<n>In the absence of a reference answer, the judge evaluates factual correctness either through its parametric knowledge or through tool augmentation.<n>We propose a risk-controlled framework that calibrates uncertainty thresholds on a held-out set so that the false discovery rate among accepted verdicts remains below a user-specified level.
  arXiv  Detail & Related papers  (2026-08-18T16:42:02Z)
• DA-RAC: Distance-Aware Calibration of LLM Judges for Trustworthy AI Auditing [6.639255145889912]
  DA-RAC is a distance-aware reference-anchored calibration method for LLM judges.<n>It improves calibration and reduces false-pass risk relative to zero-shot, chain-of-thought evaluation, and static-anchor baselines.
  arXiv  Detail & Related papers  (2026-08-15T00:29:25Z)
• LLM Judges Can Be Too Generous When There Is No Reference Answer [4.33144664431421]
  We show that judge models tend to over-credit incorrect answers in the absence of a reference answer.<n>We also show that adding reference answer information to the prompt flips the judge model's correct/incorrect decisions by as much as 85%.<n>Our results emphasize the need for calibrating the LLM judges with a sample with reference-aware evaluation before using them in reference-free setups reliably.
  arXiv  Detail & Related papers  (2026-07-14T15:31:33Z)
• Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modeling [35.945096782147864]
  MLLM judges tend to reward plausible narratives over perceptually correct answers when visual evidence conflicts with textual cues.<n>We introduce the Perceptually Perturbed Judgment dataset, which constructs minimally edited counterfactual responses.<n>We develop a unified training framework that combines a structured GRPO-based reward with a batch-ranking objective, achieving coherent global ordering without explicit pairwise labels.
  arXiv  Detail & Related papers  (2026-06-01T17:59:46Z)
• Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents? [61.49434544687523]
  We introduce REFLECT, a meta-evaluation benchmark targeting fine-grained failure detection in agentic environments.<n> REFLECT defines a detailed taxonomy of process- and outcome-level failure modes, instantiated by performing controlled and localized interventions.<n>Our experiments show that even the best-performing models achieve overall accuracies below 55% across reasoning, tool-use, and report-quality failures.
  arXiv  Detail & Related papers  (2026-05-18T23:55:08Z)
• When LLM Judges Inflate Scores: Exploring Overrating in Relevance Assessment [29.603396943658428]
  Large language models (LLMs) can be used as proxies for human judges.<n>We show that models consistently assign inflated relevance scores to passages that do not genuinely satisfy the underlying information need.<n>Experiments show that LLM-based relevance judgments can be highly sensitive to passage length and surface-level lexical cues.
  arXiv  Detail & Related papers  (2026-02-19T08:37:21Z)
• CARE: Confounder-Aware Aggregation for Reliable LLM Evaluation [21.96013518286442]
  We introduce CARE, a confounder-aware aggregation framework for judge scores.<n>We provide theoretical guarantees for identifiability and finite-sample recovery under shared confounders.<n> CARE improves aggregation accuracy, reducing error by up to 26.8%.
  arXiv  Detail & Related papers  (2026-02-09T03:17:55Z)
• Query-Document Dense Vectors for LLM Relevance Judgment Bias Analysis [4.719505127252616]
  Large Language Models (LLMs) have been used as relevance assessors for Information Retrieval (IR) evaluation collection creation.<n>We aim to understand if LLMs make systematic mistakes when judging relevance, rather than just understanding how good they are on average.<n>We introduce a clustering-based framework that embeds query-document (Q-D) pairs into a joint semantic space.
  arXiv  Detail & Related papers  (2026-01-05T03:02:33Z)
• Topic-Specific Classifiers are Better Relevance Judges than Prompted LLMs [34.14678608130442]
  The unjudged document problem is a key obstacle to the reusability of test collections in information retrieval.<n>We train topic-specific relevance classifiers by finetuning monoT5 with independent LoRA weight adaptation on the judgments of a single assessor.<n>As little as 128 initial human judgments per topic suffice to improve the comparability of models.
  arXiv  Detail & Related papers  (2025-10-06T09:38:13Z)
• Do Before You Judge: Self-Reference as a Pathway to Better LLM Evaluation [22.409170147732464]
  LLM-as-Judge frameworks are increasingly popular for AI evaluation.<n>Research findings on the relationship between models' generation and judgment abilities remain inconsistent.<n>We propose a self-reference-guided evaluation strategy that leverages a model's own answers as references.
  arXiv  Detail & Related papers  (2025-09-24T08:32:45Z)
• LLM-as-a-Judge: Rapid Evaluation of Legal Document Recommendation for Retrieval-Augmented Generation [40.06592175227558]
  This paper investigates a principled approach to evaluating Retrieval-Augmented Generation systems in legal contexts.<n>We find that traditional agreement metrics like Krippendorff's alpha can be misleading in the skewed distributions typical of AI system evaluations.<n>Our findings suggest a path toward scalable, cost-effective evaluation that maintains the precision demanded by legal applications.
  arXiv  Detail & Related papers  (2025-09-15T19:20:21Z)
• CompassJudger-2: Towards Generalist Judge Model via Verifiable Rewards [72.44810390478229]
  CompassJudger-2 is a novel generalist judge model that overcomes limitations via a task-driven, multi-domain data curation strategy.<n> CompassJudger-2 achieves superior results across multiple judge and reward benchmarks.
  arXiv  Detail & Related papers  (2025-07-12T01:34:24Z)
• RAG-Zeval: Towards Robust and Interpretable Evaluation on RAG Responses through End-to-End Rule-Guided Reasoning [64.46921169261852]
  RAG-Zeval is a novel end-to-end framework that formulates faithfulness and correctness evaluation as a rule-guided reasoning task.<n>Our approach trains evaluators with reinforcement learning, facilitating compact models to generate comprehensive and sound assessments.<n>Experiments demonstrate RAG-Zeval's superior performance, achieving the strongest correlation with human judgments.
  arXiv  Detail & Related papers  (2025-05-28T14:55:33Z)
• DAFE: LLM-Based Evaluation Through Dynamic Arbitration for Free-Form Question-Answering [12.879551933541345]
  We propose the Dynamic Arbitration Framework for Evaluation (DAFE) to evaluate large language models.<n>DAFE employs two primary LLM-as-judges and engages a third arbitrator only in cases of disagreements.<n>We show DAFE's ability to provide consistent, scalable, and resource-efficient assessments.
  arXiv  Detail & Related papers  (2025-03-11T15:29:55Z)
• RevisEval: Improving LLM-as-a-Judge via Response-Adapted References [95.29800580588592]
  RevisEval is a novel text generation evaluation paradigm via the response-adapted references.<n>RevisEval is driven by the key observation that an ideal reference should maintain the necessary relevance to the response to be evaluated.
  arXiv  Detail & Related papers  (2024-10-07T16:50:47Z)
• TrustScore: Reference-Free Evaluation of LLM Response Trustworthiness [58.721012475577716]
  Large Language Models (LLMs) have demonstrated impressive capabilities across various domains, prompting a surge in their practical applications. This paper introduces TrustScore, a framework based on the concept of Behavioral Consistency, which evaluates whether an LLMs response aligns with its intrinsic knowledge.
  arXiv  Detail & Related papers  (2024-02-19T21:12:14Z)
• Evaluate What You Can't Evaluate: Unassessable Quality for Generated Response [56.25966921370483]
  There are challenges in using reference-free evaluators based on large language models. Reference-free evaluators are more suitable for open-ended examples with different semantics responses. There are risks in using eference-free evaluators based on LLMs to evaluate the quality of dialogue responses.
  arXiv  Detail & Related papers  (2023-05-24T02:52:48Z)
• REAM$\sharp$: An Enhancement Approach to Reference-based Evaluation Metrics for Open-domain Dialog Generation [63.46331073232526]
  We present an enhancement approach to Reference-based EvAluation Metrics for open-domain dialogue systems. A prediction model is designed to estimate the reliability of the given reference set. We show how its predicted results can be helpful to augment the reference set, and thus improve the reliability of the metric.
  arXiv  Detail & Related papers  (2021-05-30T10:04:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.