Uncertainty Quantification for Regression: A Unified Framework based on kernel scores

• URL: http://arxiv.org/abs/2510.25599v1
• Date: Wed, 29 Oct 2025 15:08:41 GMT
• Title: Uncertainty Quantification for Regression: A Unified Framework based on kernel scores
• Authors: Christopher Bülte, Yusuf Sale, Gitta Kutyniok, Eyke Hüllermeier,
• Abstract summary: We introduce a family of measures for total, aleatoric, and epistemic uncertainty based on proper scoring rules.<n>We prove explicit correspondences between kernel-score characteristics and downstream behavior.<n>Experiments demonstrate that these measures are effective in downstream tasks.
• Score: 39.292428768388156
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Regression tasks, notably in safety-critical domains, require proper uncertainty quantification, yet the literature remains largely classification-focused. In this light, we introduce a family of measures for total, aleatoric, and epistemic uncertainty based on proper scoring rules, with a particular emphasis on kernel scores. The framework unifies several well-known measures and provides a principled recipe for designing new ones whose behavior, such as tail sensitivity, robustness, and out-of-distribution responsiveness, is governed by the choice of kernel. We prove explicit correspondences between kernel-score characteristics and downstream behavior, yielding concrete design guidelines for task-specific measures. Extensive experiments demonstrate that these measures are effective in downstream tasks and reveal clear trade-offs among instantiations, including robustness and out-of-distribution detection performance.

Related papers

• One Permutation Is All You Need: Fast, Reliable Variable Importance and Model Stress-Testing [0.0]
  We show that by replacing multiple random permutations with a single, deterministic and optimal permutation, we achieve a method that retains the core principles of permutation-based importance.<n>We validate this approach across nearly 200 scenarios, including real-world household finance and credit risk applications.
  arXiv  Detail & Related papers  (2025-12-15T20:50:54Z)
• Uncertainty Quantification for Regression using Proper Scoring Rules [76.24649098854219]
  We introduce a unified UQ framework for regression based on proper scoring rules, such as CRPS, logarithmic, squared error, and quadratic scores.<n>We derive closed-form expressions for the uncertainty measures under practical parametric assumptions and show how to estimate them using ensembles of models.<n>Our broad evaluation on synthetic and real-world regression datasets provides guidance for selecting reliable UQ measures.
  arXiv  Detail & Related papers  (2025-09-30T17:52:12Z)
• A Novel Framework for Uncertainty Quantification via Proper Scores for Classification and Beyond [1.5229257192293202]
  We propose a novel framework for uncertainty quantification in machine learning, which is based on proper scores.<n>Specifically, we use the kernel score, a kernel-based proper score, for evaluating sample-based generative models.<n>We generalize the calibration-sharpness decomposition beyond classification, which motivates the definition of proper calibration errors.
  arXiv  Detail & Related papers  (2025-08-25T13:11:03Z)
• Uncertainty Quantification with Proper Scoring Rules: Adjusting Measures to Prediction Tasks [19.221081896134567]
  We propose measures of uncertainty based on a known decomposition of (strictly) proper scoring rules, a specific type of loss function, into a divergence and an entropy component.<n>This leads to a flexible framework for uncertainty quantification that can be instantiated with different losses (scoring rules)<n>We show that this flexibility is indeed advantageous. In particular, we analyze the task of selective prediction and show that the scoring rule should ideally match the task loss.
  arXiv  Detail & Related papers  (2025-05-28T16:22:53Z)
• Advancing Embodied Agent Security: From Safety Benchmarks to Input Moderation [52.83870601473094]
  Embodied agents exhibit immense potential across a multitude of domains.<n>Existing research predominantly concentrates on the security of general large language models.<n>This paper introduces a novel input moderation framework, meticulously designed to safeguard embodied agents.
  arXiv  Detail & Related papers  (2025-04-22T08:34:35Z)
• SConU: Selective Conformal Uncertainty in Large Language Models [59.25881667640868]
  We propose a novel approach termed Selective Conformal Uncertainty (SConU)<n>We develop two conformal p-values that are instrumental in determining whether a given sample deviates from the uncertainty distribution of the calibration set at a specific manageable risk level.<n>Our approach not only facilitates rigorous management of miscoverage rates across both single-domain and interdisciplinary contexts, but also enhances the efficiency of predictions.
  arXiv  Detail & Related papers  (2025-04-19T03:01:45Z)
• Segmentation Re-thinking Uncertainty Estimation Metrics for Semantic Segmentation [12.532289778772185]
  semantic segmentation is a fundamental application within machine learning. The metric known as PAvPU (Patch Accuracy versus Patch Uncertainty) has been developed as a specialized tool for evaluating entropy-based uncertainty in image segmentation tasks. Our investigation identifies three core deficiencies within the PAvPU framework and proposes robust solutions.
  arXiv  Detail & Related papers  (2024-03-28T20:34:02Z)
• From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimation [5.355925496689674]
  We build a framework that allows one to generate different predictive uncertainty measures.<n>We validate our method on image datasets by evaluating its performance in detecting out-of-distribution and misclassified instances.
  arXiv  Detail & Related papers  (2024-02-16T14:40:22Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• dugMatting: Decomposed-Uncertainty-Guided Matting [83.71273621169404]
  We propose a decomposed-uncertainty-guided matting algorithm, which explores the explicitly decomposed uncertainties to efficiently and effectively improve the results. The proposed matting framework relieves the requirement for users to determine the interaction areas by using simple and efficient labeling.
  arXiv  Detail & Related papers  (2023-06-02T11:19:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.