Evaluating and Calibrating LLM Confidence on Questions with Multiple Correct Answers

• URL: http://arxiv.org/abs/2602.07842v1
• Date: Sun, 08 Feb 2026 07:03:27 GMT
• Title: Evaluating and Calibrating LLM Confidence on Questions with Multiple Correct Answers
• Abstract summary: We show that existing training-free methods break down in the presence of multiple valid answers.<n>We propose Semantic Confidence Aggregation (SCA), which aggregates confidence over multiple high-probability sampled responses.<n> SCA achieves state-of-the-art calibration performance under mixed-answer settings while preserving strong calibration on single-answer questions.
• Score: 11.832359446987551
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Confidence calibration is essential for making large language models (LLMs) reliable, yet existing training-free methods have been primarily studied under single-answer question answering. In this paper, we show that these methods break down in the presence of multiple valid answers, where disagreement among equally correct responses leads to systematic underestimation of confidence. To enable a systematic study of this phenomenon, we introduce MACE, a benchmark of 12,000 factual questions spanning six domains with varying numbers of correct answers. Experiments across 15 representative calibration methods and four LLM families (7B-72B) reveal that while accuracy increases with answer cardinality, estimated confidence consistently decreases, causing severe miscalibration for questions with mixed answer counts. To address this issue, we propose Semantic Confidence Aggregation (SCA), which aggregates confidence over multiple high-probability sampled responses. SCA achieves state-of-the-art calibration performance under mixed-answer settings while preserving strong calibration on single-answer questions.

Related papers

• Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models [7.714704554667688]
  Large Language Models (LLMs) increasingly support decision-making in high-stakes domains.<n>LLMs often hallucinate and express confidence that is misaligned with factual correctness.<n>We study claim-level confidence calibration as a decision-relevant uncertainty signal.
  arXiv  Detail & Related papers  (2026-08-23T16:09:54Z)
• DirEAG: Dirichlet Evidence Aggregation for Calibrating Verbalized Confidence in Mathematical Reasoning [2.867517731896504]
  Black-box verbalized confidence is difficult to calibrate in large language models.<n>We propose DirEAG, a Dirichlet Evidence Aggregation method that converts each elicited answer-confidence observation into calibrated soft evidence.
  arXiv  Detail & Related papers  (2026-08-21T03:48:25Z)
• Confidence Before Answering: A Paradigm Shift for Efficient LLM Uncertainty Estimation [47.91529693614168]
  Existing methods are predominantly answer-first, producing confidence only after generating an answer.<n>We study a confidence-first paradigm, where the model outputs its confidence before answering, interpreting this score as the model's probability of answering the question correctly.<n>We propose CoCA, a reinforcement learning framework that jointly optimize confidence calibration and answer accuracy via segmented credit assignment.
  arXiv  Detail & Related papers  (2026-03-06T04:03:13Z)
• On Calibration of Large Language Models: From Response To Capability [66.59139960234326]
  Large language models (LLMs) are widely deployed as general-purpose problem solvers.<n>We introduce capability calibration, which targets the model's expected accuracy on a query.<n>Our results demonstrate that capability-calibrated confidence improves pass@$k$ prediction and inference budget allocation.
  arXiv  Detail & Related papers  (2026-02-14T01:07:45Z)
• Calibration Is Not Enough: Evaluating Confidence Estimation Under Language Variations [49.84786015324238]
  Confidence estimation (CE) indicates how reliable the answers of large language models (LLMs) are, and can impact user trust and decision-making.<n>We present a comprehensive evaluation framework for CE that measures their confidence quality on three new aspects.<n>These include robustness of confidence against prompt perturbations, stability across semantic equivalent answers, and sensitivity to semantically different answers.
  arXiv  Detail & Related papers  (2026-01-12T23:16:50Z)
• Refine and Align: Confidence Calibration through Multi-Agent Interaction in VQA [8.879640496857661]
  We introduce AlignVQA, a debate-based multi-agent framework for Visual Question Answering.<n>We find that more calibrated specialized agents produce better aligned confidences.<n>We also introduce a novel differentiable calibration-aware loss function called aligncal to fine-tune the specialized agents.
  arXiv  Detail & Related papers  (2025-11-14T11:08:21Z)
• Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding [48.92310906093414]
  We introduce a novel approach for calibrating uncertainty quantification (UQ) tailored for multi-modal large language models (LLMs)<n>We leverage cross-modal consistency in addition to self-consistency to improve the calibration of the multi-modal models.<n>We evaluate the proposed approach across multiple multi-modal tasks, such as medical question answering (Slake) and visual question answering (VQAv2), considering multi-modal models such as LLaVA-Med and LLaVA.
  arXiv  Detail & Related papers  (2025-04-30T19:19:21Z)
• Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models [34.59785123314865]
  A safe and trustworthy use of Large Language Models (LLMs) requires an accurate expression of confidence in their answers.<n>We propose a novel Reinforcement Learning approach that allows to directly fine-tune LLMs to express calibrated confidence estimates alongside their answers to factual questions.
  arXiv  Detail & Related papers  (2025-03-04T13:48:50Z)
• Fact-Level Confidence Calibration and Self-Correction [64.40105513819272]
  We propose a Fact-Level framework that calibrates confidence to relevance-weighted correctness at the fact level. We also develop Confidence-Guided Fact-level Self-Correction ($textbfConFix$), which uses high-confidence facts within a response as additional knowledge to improve low-confidence ones.
  arXiv  Detail & Related papers  (2024-11-20T14:15:18Z)
• Graph-based Confidence Calibration for Large Language Models [22.394717844099684]
  We propose using an auxiliary learning model to assess response correctness based on the self-consistency of multiple outputs generated by the large language models.<n>Our method builds a consistency graph to represent the agreement among multiple responses and uses a graph neural network (GNN) to estimate the likelihood that each response is correct.
  arXiv  Detail & Related papers  (2024-11-03T20:36:44Z)
• QA-Calibration of Language Model Confidence Scores [31.04440851437498]
  generative question-and-answering systems need to provide well-calibrated confidence scores.<n>Existing calibration methods aim to ensure that the confidence score is, *on average*, indicative of the likelihood that the answer is correct.<n>We introduce QA-calibration, which ensures calibration holds across different question-and-answer groups.
  arXiv  Detail & Related papers  (2024-10-09T07:12:24Z)
• Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflection [90.71323430635593]
  We propose a novel self-detection paradigm that considers the comprehensive answer space beyond LLM-generated answers. Building upon this paradigm, we introduce a two-step framework, which firstly instructs LLM to reflect and provide justifications for each candidate answer. This framework can be seamlessly integrated with existing approaches for superior self-detection.
  arXiv  Detail & Related papers  (2024-03-15T02:38:26Z)
• Fact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models [84.94220787791389]
  We propose Fact-and-Reflection (FaR) prompting, which improves the LLM calibration in two steps. Experiments show that FaR achieves significantly better calibration; it lowers the Expected Error by 23.5%. FaR even elicits the capability of verbally expressing concerns in less confident scenarios.
  arXiv  Detail & Related papers  (2024-02-27T01:37:23Z)
• Reconfidencing LLMs from the Grouping Loss Perspective [56.801251926946485]
  Large Language Models (LLMs) are susceptible to generating hallucinated answers in a confident tone. Recent findings show that controlling uncertainty must go beyond calibration. We construct a new evaluation dataset derived from a knowledge base to assess confidence scores given to answers of Mistral and LLaMA.
  arXiv  Detail & Related papers  (2024-02-07T15:40:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.