When Can We Trust LLM Graders? Calibrating Confidence for Automated Assessment

• URL: http://arxiv.org/abs/2603.29559v1
• Date: Tue, 31 Mar 2026 10:38:22 GMT
• Title: When Can We Trust LLM Graders? Calibrating Confidence for Automated Assessment
• Abstract summary: Large Language Models (LLMs) show promise for automated grading, but their outputs can be unreliable.<n>This enables selective automation where high-confidence predictions are processed automatically while uncertain cases are flagged for human review.
• Score: 1.896929753246251
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) show promise for automated grading, but their outputs can be unreliable. Rather than improving grading accuracy directly, we address a complementary problem: \textit{predicting when an LLM grader is likely to be correct}. This enables selective automation where high-confidence predictions are processed automatically while uncertain cases are flagged for human review. We compare three confidence estimation methods (self-reported confidence, self-consistency voting, and token probability) across seven LLMs of varying scale (4B to 120B parameters) on three educational datasets: RiceChem (long-answer chemistry), SciEntsBank, and Beetle (short-answer science). Our experiments reveal that self-reported confidence consistently achieves the best calibration across all conditions (avg ECE 0.166 vs 0.229 for self-consistency). Surprisingly, self-consistency remains 38\% worse despite requiring 5$\times$ the inference cost. Larger models exhibit substantially better calibration though gains vary by dataset and method (e.g., a 28\% ECE reduction for self-reported), with GPT-OSS-120B achieving the best calibration (avg ECE 0.100) and strong discrimination (avg AUC 0.668). We also observe that confidence is strongly top-skewed across methods, creating a ``confidence floor'' that practitioners must account for when setting thresholds. These findings suggest that simply asking LLMs to report their confidence provides a practical approach for identifying reliable grading predictions. Code is available \href{https://github.com/sonkar-lab/llm_grading_calibration}{here}.

Related papers

• ConfidenceBench: Evaluating Confidence Calibration in Large Language Models [14.831793255747181]
  ConfidenceBench is a calibration benchmark for large language models.<n>It evaluates verbalized confidence estimates in 15 frontier large language models.<n> Confidence is elicited via prompting, requiring no access to model logits.
  arXiv  Detail & Related papers  (2026-07-10T01:56:30Z)
• CaliDist: Calibrating Large Language Models via Behavioral Robustness to Distraction [51.56484100374058]
  Existing calibration methods for Large Language Models (LLMs) often overlook a critical dimension of trustworthiness: a model's behavioral robustness.<n>We introduce textscCaliDist, a novel post-hoc calibration approach that directly measures and penalizes a model's susceptibility to distraction.<n>textscCaliDist consistently achieves lower Expected Error (ECE) and Brier Score compared with strong baselines.
  arXiv  Detail & Related papers  (2026-06-04T07:27:53Z)
• The Illusion of Certainty: Decoupling Capability and Calibration in On-Policy Distillation [67.26315138466312]
  We propose a calibration-aware OPD framework, CaOPD, that estimates empirical confidence from model rollouts.<n>Our findings highlight that capability distillation does not imply confidence, and that confidence should be treated as an essential objective in post-training.
  arXiv  Detail & Related papers  (2026-04-18T04:43:40Z)
• Fine-grained Approaches for Confidence Calibration of LLMs in Automated Code Revision [16.289117637700446]
  A canonical mitigation method is to provide calibrated confidence scores that faithfully reflect their likelihood of correctness at the instance-level.<n>Our study proposes local Platt-scaling applied separately to three different fine-grained confidence scores.<n>We find that fine-grained confidence scores consistently achieve lower calibration error across a broader range of probability intervals.
  arXiv  Detail & Related papers  (2026-04-08T06:41:09Z)
• CHiL(L)Grader: Calibrated Human-in-the-Loop Short-Answer Grading [1.5829605001132387]
  We introduce CHiL(L)Grader, the first automated grading framework that incorporates calibrated confidence estimation into a human-in-the-loop workflow.<n>CHiL(L)Grader automates only high-confidence predictions while routing uncertain cases to human graders, and adapts to evolvings and unseen questions.<n>A QWK gap of 0.347 between accepted and rejected predictions confirms the effectiveness of the confidence-based routing.
  arXiv  Detail & Related papers  (2026-03-12T14:04:58Z)
• Know When You're Wrong: Aligning Confidence with Correctness for LLM Error Detection [0.0]
  Large language models (LLMs) are increasingly deployed in critical decision-making systems.<n>We introduce a normalized confidence score based on output anchor token probabilities.<n>This enables direct detection of errors and hallucinations with minimal overhead.
  arXiv  Detail & Related papers  (2026-02-18T07:05:12Z)
• Annotation-Efficient Universal Honesty Alignment [70.05453324928955]
  Existing methods either rely on training-free confidence estimation or training-based calibration with correctness annotations.<n>Elicitation-Then-Calibration (EliCal) is a two-stage framework that first elicits internal confidence using inexpensive self-consistency supervision, then calibrates this confidence with a small set of correctness annotations.<n>EliCal achieves near-optimal alignment with only 1k correctness annotations (0.18% of full supervision) and better alignment performance on unseen MMLU tasks than the calibration-only baseline.
  arXiv  Detail & Related papers  (2025-10-20T13:05:22Z)
• Judging with Confidence: Calibrating Autoraters to Preference Distributions [56.17041629492863]
  We argue that a reliable autorater must learn to model the full distribution of preferences defined by a target population.<n>We present two learning methods tailored to different data conditions.<n>Our results show that finetuning autoraters with a distribution-matching objective leads to verbalized probability predictions that are better aligned with the target preference distribution.
  arXiv  Detail & Related papers  (2025-09-30T20:36:41Z)
• ConfTuner: Training Large Language Models to Express Their Confidence Verbally [58.63318088243125]
  Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as science, law, and healthcare.<n>LLMs are often observed to generate incorrect answers with high confidence, a phenomenon known as "overconfidence"
  arXiv  Detail & Related papers  (2025-08-26T09:25:32Z)
• The Confidence Paradox: Can LLM Know When It's Wrong [5.445980143646736]
  Document Visual Question Answering (DocVQA) models often produce overconfident or ethically misaligned responses.<n>We propose HonestVQA, a model-agnostic, self-supervised framework that aligns model confidence with correctness using weighted loss and contrastive learning.
  arXiv  Detail & Related papers  (2025-06-30T02:06:54Z)
• Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator [20.81467363927836]
  Post-trained language models (PoLMs) often suffer from over-confidence, assigning high confidence to both correct and incorrect outputs.<n>A major obstacle in calibrating PoLMs is the scarcity of labeled data for individual downstream tasks.<n>We propose Disagreement-Aware Confidence Alignment (DACA) to optimize parameters in post-hoc confidence calibration.
  arXiv  Detail & Related papers  (2025-05-22T13:55:39Z)
• LACIE: Listener-Aware Finetuning for Confidence Calibration in Large Language Models [69.68379406317682]
  We introduce a listener-aware finetuning method (LACIE) to calibrate implicit and explicit confidence markers. We show that LACIE models the listener, considering not only whether an answer is right, but whether it will be accepted by a listener. We find that training with LACIE results in 47% fewer incorrect answers being accepted while maintaining the same level of acceptance for correct answers.
  arXiv  Detail & Related papers  (2024-05-31T17:16:38Z)
• Pearls from Pebbles: Improved Confidence Functions for Auto-labeling [51.44986105969375]
  threshold-based auto-labeling (TBAL) works by finding a threshold on a model's confidence scores above which it can accurately label unlabeled data points. We propose a framework for studying the emphoptimal TBAL confidence function. We develop a new post-hoc method specifically designed to maximize performance in TBAL systems.
  arXiv  Detail & Related papers  (2024-04-24T20:22:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.