The ACUTE Protocol: Operationalizing Language Model Activations for Better Calibration, Utility, and Trust

• URL: http://arxiv.org/abs/2606.07822v1
• Date: Fri, 05 Jun 2026 20:15:50 GMT
• Title: The ACUTE Protocol: Operationalizing Language Model Activations for Better Calibration, Utility, and Trust
• Abstract summary: We develop a new metric, expected utility renormalized by the oracle (EURO), that balances calibration and informativeness.<n>We also propose a general-purpose-based confidence, utility, and trust estimation protocol (ACUTE) to appropriately adjudicate uncertainty.
• Score: 39.08747366439304
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As language models improve and become increasingly deployed to solve a variety of tasks, trustworthiness becomes essential. Calibration is a good proxy for trust: well-calibrated confidence estimates help inform the risk versus reward tradeoff when trusting a specific model output. Unfortunately, even as models improve, they remain poorly calibrated, often biasing towards overconfidence. Additionally, calibration can be gamed: a policy that always predicts the base rate is perfectly calibrated, but completely uninformative. To resolve this, we develop a new metric, expected utility renormalized by the oracle (EURO), that balances calibration and informativeness. We also propose a general-purpose activation-based confidence, utility, and trust estimation protocol (ACUTE) to appropriately adjudicate uncertainty. The ACUTE protocol provides flexible, sample-efficient, and compute-efficient confidence estimators for 3 tasks including multiple choice question answering, tool-calling, and scientific document summarization across 6 models from 4 model families. ACUTE outperforms strong baselines on EURO, while maintaining low calibration error. Taken together, our work shows that equipping LLMs with the ACUTE protocol can improve calibration, utility, and trustworthiness in numerous settings.

Related papers

• An Empirical Study of Security Calibration in Large Language Models for Code [5.167904179040144]
  We present the first large-scale empirical study of security calibration in LLM-generated code.<n>We evaluate GPT-4o-mini, Gemini-2.0-Flash, and Qwen3-Coder-Next across multiple benchmarks.<n>Our results suggest that overconfidence is prevalent across the evaluated LLMs.
  arXiv  Detail & Related papers  (2026-06-30T05:37:39Z)
• CaliDist: Calibrating Large Language Models via Behavioral Robustness to Distraction [51.56484100374058]
  Existing calibration methods for Large Language Models (LLMs) often overlook a critical dimension of trustworthiness: a model's behavioral robustness.<n>We introduce textscCaliDist, a novel post-hoc calibration approach that directly measures and penalizes a model's susceptibility to distraction.<n>textscCaliDist consistently achieves lower Expected Error (ECE) and Brier Score compared with strong baselines.
  arXiv  Detail & Related papers  (2026-06-04T07:27:53Z)
• Process Supervision of Confidence Margin for Calibrated LLM Reasoning [52.373121066425455]
  Scaling test-time computation with reinforcement learning (RL) has emerged as a reliable path to improve large language models (LLM) reasoning ability.<n>Yet, outcome-based reward often incentivizes models to be overconfident, leading to hallucinations, unreliable confidence-based control, and unnecessary compute allocation.<n>We introduce Reinforcement Learning with Confidence Margin (textbfRLCM), a calibration-aware RL framework that jointly optimize correctness and confidence reliability.
  arXiv  Detail & Related papers  (2026-04-25T14:40:13Z)
• Fine-grained Approaches for Confidence Calibration of LLMs in Automated Code Revision [16.289117637700446]
  A canonical mitigation method is to provide calibrated confidence scores that faithfully reflect their likelihood of correctness at the instance-level.<n>Our study proposes local Platt-scaling applied separately to three different fine-grained confidence scores.<n>We find that fine-grained confidence scores consistently achieve lower calibration error across a broader range of probability intervals.
  arXiv  Detail & Related papers  (2026-04-08T06:41:09Z)
• On Calibration of Large Language Models: From Response To Capability [66.59139960234326]
  Large language models (LLMs) are widely deployed as general-purpose problem solvers.<n>We introduce capability calibration, which targets the model's expected accuracy on a query.<n>Our results demonstrate that capability-calibrated confidence improves pass@$k$ prediction and inference budget allocation.
  arXiv  Detail & Related papers  (2026-02-14T01:07:45Z)
• Balancing Classification and Calibration Performance in Decision-Making LLMs via Calibration Aware Reinforcement Learning [10.123352394689134]
  Well-calibrated confidence enables downstream systems to decide when to trust a model and when to defer to fallback mechanisms.<n>We show that while RLVR improves task performance, it produces extremely overconfident models.<n>We propose a calibration-aware reinforcement learning formulation that directly adjusts decision-token probabilities.
  arXiv  Detail & Related papers  (2026-01-19T18:31:31Z)
• ConfTuner: Training Large Language Models to Express Their Confidence Verbally [58.63318088243125]
  Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as science, law, and healthcare.<n>LLMs are often observed to generate incorrect answers with high confidence, a phenomenon known as "overconfidence"
  arXiv  Detail & Related papers  (2025-08-26T09:25:32Z)
• Calibrating Large Language Models with Sample Consistency [76.23956851098598]
  We explore the potential of deriving confidence from the distribution of multiple randomly sampled model generations, via three measures of consistency. Results show that consistency-based calibration methods outperform existing post-hoc approaches. We offer practical guidance on choosing suitable consistency metrics for calibration, tailored to the characteristics of various LMs.
  arXiv  Detail & Related papers  (2024-02-21T16:15:20Z)
• Calibrating Long-form Generations from Large Language Models [34.72041258464477]
  Large Language Models' (LLMs) confidence scores should align with the actual likelihood of its responses being correct. Current confidence elicitation methods and calibration metrics rely on a binary true/false assessment of response correctness. We introduce a unified calibration framework, in which both the correctness of the LLMs' responses and their associated confidence levels are treated as distributions across a range of scores.
  arXiv  Detail & Related papers  (2024-02-09T17:00:32Z)
• A Close Look into the Calibration of Pre-trained Language Models [56.998539510508515]
  Pre-trained language models (PLMs) may fail in giving reliable estimates of their predictive uncertainty. We study the dynamic change in PLMs' calibration performance in training. We extend two recently proposed learnable methods that directly collect data to train models to have reasonable confidence estimations.
  arXiv  Detail & Related papers  (2022-10-31T21:31:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.