Quantification of Credal Uncertainty: A Distance-Based Approach

• URL: http://arxiv.org/abs/2603.27270v1
• Date: Sat, 28 Mar 2026 13:50:48 GMT
• Title: Quantification of Credal Uncertainty: A Distance-Based Approach
• Abstract summary: We introduce a family of uncertainty measures within the framework of Integral Probability Metrics (IPMs)<n>We instantiate the framework with the total variation distance and obtain simple, efficient uncertainty measures for multiclass classification.<n> Empirical results confirm practical usefulness, with favorable performance at low computational cost.
• Score: 34.89687688300613
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Credal sets, i.e., closed convex sets of probability measures, provide a natural framework to represent aleatoric and epistemic uncertainty in machine learning. Yet how to quantify these two types of uncertainty for a given credal set, particularly in multiclass classification, remains underexplored. In this paper, we propose a distance-based approach to quantify total, aleatoric, and epistemic uncertainty for credal sets. Concretely, we introduce a family of such measures within the framework of Integral Probability Metrics (IPMs). The resulting quantities admit clear semantic interpretations, satisfy natural theoretical desiderata, and remain computationally tractable for common choices of IPMs. We instantiate the framework with the total variation distance and obtain simple, efficient uncertainty measures for multiclass classification. In the binary case, this choice recovers established uncertainty measures, for which a principled multiclass generalization has so far been missing. Empirical results confirm practical usefulness, with favorable performance at low computational cost.

Related papers

• Efficient Credal Prediction through Decalibration [27.17944683306638]
  We propose an efficient method for credal prediction that is grounded in the notion of relative likelihood.<n>We show that our method yields credal sets with strong performance across diverse tasks.<n> Notably, we demonstrate credal prediction on models such as TabPFN and CLIP.
  arXiv  Detail & Related papers  (2026-03-09T15:30:10Z)
• Credal Ensemble Distillation for Uncertainty Quantification [12.36665123584814]
  We propose credal ensemble distillation (CED), a framework that compresses a deep ensemble into a single model, CREDIT, for classification tasks.<n>CED achieves superior or comparable uncertainty estimation compared to several existing baselines, while substantially reducing inference overhead compared to deep ensembles.
  arXiv  Detail & Related papers  (2025-11-14T14:53:42Z)
• Towards Reliable LLM-based Robot Planning via Combined Uncertainty Estimation [68.106428321492]
  Large language models (LLMs) demonstrate advanced reasoning abilities, enabling robots to understand natural language instructions and generate high-level plans with appropriate grounding.<n>LLMs hallucinations present a significant challenge, often leading to overconfident yet potentially misaligned or unsafe plans.<n>We present Combined Uncertainty estimation for Reliable Embodied planning (CURE), which decomposes the uncertainty into epistemic and intrinsic uncertainty, each estimated separately.
  arXiv  Detail & Related papers  (2025-10-09T10:26:58Z)
• Fine-Grained Uncertainty Decomposition in Large Language Models: A Spectral Approach [32.528332797693984]
  We introduce Spectral Uncertainty, a novel approach to quantifying and decomposing uncertainties in Large Language Models.<n>Unlike existing baseline methods, our approach incorporates a fine-grained representation of semantic similarity.<n> Empirical evaluations demonstrate that Spectral Uncertainty outperforms state-of-the-art methods in estimating both aleatoric and total uncertainty.
  arXiv  Detail & Related papers  (2025-09-26T12:39:10Z)
• SConU: Selective Conformal Uncertainty in Large Language Models [59.25881667640868]
  We propose a novel approach termed Selective Conformal Uncertainty (SConU)<n>We develop two conformal p-values that are instrumental in determining whether a given sample deviates from the uncertainty distribution of the calibration set at a specific manageable risk level.<n>Our approach not only facilitates rigorous management of miscoverage rates across both single-domain and interdisciplinary contexts, but also enhances the efficiency of predictions.
  arXiv  Detail & Related papers  (2025-04-19T03:01:45Z)
• CLUE: Concept-Level Uncertainty Estimation for Large Language Models [49.92690111618016]
  We propose a novel framework for Concept-Level Uncertainty Estimation for Large Language Models (LLMs) We leverage LLMs to convert output sequences into concept-level representations, breaking down sequences into individual concepts and measuring the uncertainty of each concept separately. We conduct experiments to demonstrate that CLUE can provide more interpretable uncertainty estimation results compared with sentence-level uncertainty.
  arXiv  Detail & Related papers  (2024-09-04T18:27:12Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• Quantifying Aleatoric and Epistemic Uncertainty with Proper Scoring Rules [19.221081896134567]
  Uncertainty representation and quantification are paramount in machine learning. We propose measures for the quantification of aleatoric and (epistemic) uncertainty based on proper scoring rules.
  arXiv  Detail & Related papers  (2024-04-18T14:20:19Z)
• Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling [69.83976050879318]
  In large language models (LLMs), identifying sources of uncertainty is an important step toward improving reliability, trustworthiness, and interpretability. In this paper, we introduce an uncertainty decomposition framework for LLMs, called input clarification ensembling. Our approach generates a set of clarifications for the input, feeds them into an LLM, and ensembles the corresponding predictions.
  arXiv  Detail & Related papers  (2023-11-15T05:58:35Z)
• What is Flagged in Uncertainty Quantification? Latent Density Models for Uncertainty Categorization [68.15353480798244]
  Uncertainty Quantification (UQ) is essential for creating trustworthy machine learning models. Recent years have seen a steep rise in UQ methods that can flag suspicious examples. We propose a framework for categorizing uncertain examples flagged by UQ methods in classification tasks.
  arXiv  Detail & Related papers  (2022-07-11T19:47:00Z)
• Ensemble-based Uncertainty Quantification: Bayesian versus Credal Inference [0.0]
  We consider ensemble-based approaches to uncertainty quantification. We specifically focus on Bayesian methods and approaches based on so-called credal sets. The effectiveness of corresponding measures is evaluated and compared in an empirical study on classification with a reject option.
  arXiv  Detail & Related papers  (2021-07-21T22:47:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.