On the QUEST for Uncertainty Quantification via Highest Density Regions

• URL: http://arxiv.org/abs/2606.19569v1
• Date: Wed, 17 Jun 2026 20:13:45 GMT
• Title: On the QUEST for Uncertainty Quantification via Highest Density Regions
• Abstract summary: Uncertainty quantification (UQ) is essential for reliable decision-making in safety-critical applications in machine learning.<n>We propose an alternative framework, in which uncertainty is characterised by the volume of the most probable subset of a distribution's support.<n>QUEST (Quantifying Uncertainty via highest dEnSiTy regions) is a novel approach to UQ based on the concentration of Lebesgue measure at a distribution's peak(s)
• Score: 5.1459761018260854
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Uncertainty quantification (UQ) is essential for reliable decision-making in safety-critical applications in probabilistic machine learning. For regression problems, dominant scalar UQ approaches - notably, those based on proper scoring rules - measure uncertainty via pointwise predictive risk. This can lead to counterintuitive results when the target statistic is not the conditional expectation. We propose an alternative framework, in which uncertainty is characterised by the volume of the most probable subset of a distribution's support. QUEST (Quantifying Uncertainty via highest dEnSiTy regions) is a novel approach to UQ based on the concentration of Lebesgue measure at a distribution's peak(s), evaluated at one or more values of a robustness parameter $α$. We establish connections between our measures and classical statistics from information theory and economics. We show that, unlike popular alternatives based on proper scoring rules, QUEST measures of epistemic and aleatoric uncertainty satisfy a set of axioms adapted from the UQ literature, including monotonicity under distributional spread and invariance to location shifts. Selective prediction benchmarks confirm that QUEST performs favourably against standard measures such as variance and differential entropy.

Related papers

• An Isotropic Approach to Efficient Uncertainty Quantification with Gradient Norms [7.439615448864818]
  We propose a lightweight method for quantifying predictive uncertainty in neural networks.<n>We use two approximations: a first-order Taylor expansion that expresses uncertainty in terms of the gradient of the prediction, and an isotropy assumption on the parameter covariance.<n>We then use the estimates to investigate when each uncertainty type carries useful signal for predicting answer correctness in question answering with large language models.
  arXiv  Detail & Related papers  (2026-03-31T09:13:09Z)
• Entropy Alone is Insufficient for Safe Selective Prediction in LLMs [20.664633053172327]
  Selective prediction systems can mitigate harms resulting from language model hallucinations by abstaining from answering in high-risk cases.<n>Uncertainty quantification techniques are often employed to identify such cases, but are rarely evaluated in the context of the wider selective prediction policy.<n>We identify a model-dependent failure mode of entropy-based uncertainty methods that leads to unreliable abstention behaviour, and address it by combining entropy scores with a correctness probe signal.
  arXiv  Detail & Related papers  (2026-03-22T11:27:13Z)
• Uncertainty as Feature Gaps: Epistemic Uncertainty Quantification of LLMs in Contextual Question-Answering [29.4458902836278]
  We introduce a task-agnostic, token-level uncertainty measure defined as the cross-entropy between the predictive distribution of the given model and the unknown true distribution.<n>We derive an upper bound for uncertainty and show that it can be interpreted as semantic feature gaps in the given model's hidden representations.<n>We apply this generic framework to the contextual QA task and hypothesize that three features approximate this gap: context-reliance, context comprehension, and honesty.
  arXiv  Detail & Related papers  (2025-10-03T02:09:25Z)
• Uncertainty Quantification for Regression using Proper Scoring Rules [76.24649098854219]
  We introduce a unified UQ framework for regression based on proper scoring rules, such as CRPS, logarithmic, squared error, and quadratic scores.<n>We derive closed-form expressions for the uncertainty measures under practical parametric assumptions and show how to estimate them using ensembles of models.<n>Our broad evaluation on synthetic and real-world regression datasets provides guidance for selecting reliable UQ measures.
  arXiv  Detail & Related papers  (2025-09-30T17:52:12Z)
• Multidimensional Uncertainty Quantification via Optimal Transport [87.97146725546502]
  We take a multidimensional view on uncertainty quantification (UQ) by stacking complementary UQ measures into a vector.<n>VecUQ-OT shows high efficiency even when individual measures fail.
  arXiv  Detail & Related papers  (2025-09-26T14:09:03Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• SConU: Selective Conformal Uncertainty in Large Language Models [59.25881667640868]
  We propose a novel approach termed Selective Conformal Uncertainty (SConU)<n>We develop two conformal p-values that are instrumental in determining whether a given sample deviates from the uncertainty distribution of the calibration set at a specific manageable risk level.<n>Our approach not only facilitates rigorous management of miscoverage rates across both single-domain and interdisciplinary contexts, but also enhances the efficiency of predictions.
  arXiv  Detail & Related papers  (2025-04-19T03:01:45Z)
• Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model [68.34559610536614]
  We argue that the aleatoric uncertainty is an inherent attribute of the data and can only be correctly estimated with an unbiased oracle model. We propose a new sampling and selection strategy at train time to approximate the oracle model for aleatoric uncertainty estimation. Our results show that our solution achieves both accurate deterministic results and reliable uncertainty estimation.
  arXiv  Detail & Related papers  (2021-11-22T08:54:10Z)
• Distribution-free uncertainty quantification for classification under label shift [105.27463615756733]
  We focus on uncertainty quantification (UQ) for classification problems via two avenues. We first argue that label shift hurts UQ, by showing degradation in coverage and calibration. We examine these techniques theoretically in a distribution-free framework and demonstrate their excellent practical performance.
  arXiv  Detail & Related papers  (2021-03-04T20:51:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.