Uncertainty quantification via conformal prediction in data assimilation

• URL: http://arxiv.org/abs/2606.27001v1
• Date: Thu, 25 Jun 2026 13:18:15 GMT
• Title: Uncertainty quantification via conformal prediction in data assimilation
• Abstract summary: conformal prediction (CP) is a recent machine learning (ML) method to quantify uncertainty in a controlled, idealized setting.<n>We compare and evaluate the average empirical coverage, the average interval length, miss low, miss high and average interval score loss (AISL) for three variants of CP.
• Score: 26.03725296495402
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quantifying the evolution of uncertainty is critical to both probabilistic forecasting and data assimilation in numerical weather prediction. In this study, we investigate the applicability of conformal prediction (CP), a recent machine learning (ML) method, to quantify uncertainty in a controlled, idealized setting. We use the one dimensional modified shallow water model, designed to mimic the convective process. CP provides a set of possible outcomes with a chosen confidence level. Here, we compare and evaluate the average empirical coverage, the average interval length, miss low, miss high and average interval score loss (AISL) for three variants of CP, namely a) Standard CP, b) Normalized CP and c) Conformalized Quantile Regression. We further compare these CP-based uncertainty estimates with traditional ensemble-based measures such as standard deviation intervals and ensemble spread. In addition, we investigate the integration of CP-derived uncertainty within the data assimilation cycle through CP perturbations. Our results highlight the strengths and limitations of each approach, providing insight into the effectiveness of CP to complement common ensemble-based uncertainty quantification in simplified atmospheric models.

Related papers

• Probabilistic Object Detection with Conformal Prediction [2.21658458251407]
  Conformal Prediction (CP) is a distribution-free method for constructing prediction sets with marginal finite-sample coverage guarantees.<n>Standard, unscaled CP produces fixed-width prediction intervals across inputs, leading to unnecessary width for low-uncertainty predictions.<n>We show that scaled CP consistently improves interval sharpness over unscaled CP, achieving up to 19% higher IoU and 39% lower interval scores, without sacrificing coverage.
  arXiv  Detail & Related papers  (2026-05-08T10:23:43Z)
• Reliable Hierarchical Operating System Fingerprinting via Conformal Prediction [62.40452053128524]
  Conformal Prediction (CP) could be wrapped around existing methods to obtain prediction sets with guaranteed coverage.<n>This work addresses these limitations by introducing and evaluating two distinct structured CP strategies.<n>While both methods satisfy validity guarantees, they expose a fundamental trade-off between level-wise efficiency and structural consistency.
  arXiv  Detail & Related papers  (2026-02-13T11:20:48Z)
• Quantifying Epistemic Predictive Uncertainty in Conformal Prediction [11.09458914721516]
  We study the problem of quantifying uncertainty faced at prediction time due to the existence of multiple plausible predictive models.<n>We build on recent results showing that, under a mild assumption, any full conformal prediction procedure induces a set of closed and convex predictive distributions.<n>We propose a computationally efficient and analytically tractable uncertainty measure, based on emphMaximum Mean Imprecision, to quantify the uncertainty.
  arXiv  Detail & Related papers  (2026-02-02T05:38:07Z)
• Bridging the Gap Between Bayesian Deep Learning and Ensemble Weather Forecasts [100.26854618129039]
  Weather forecasting is fundamentally challenged by the chaotic nature of the atmosphere.<n>Recent advances in Bayesian Deep Learning (BDL) offer a promising but often disconnected alternative.<n>We bridge these paradigms through a unified hybrid BDL framework for ensemble weather forecasting.
  arXiv  Detail & Related papers  (2025-11-18T07:49:52Z)
• Conformal Prediction for Uncertainty Estimation in Drug-Target Interaction Prediction [1.474945380093949]
  We analyze three cluster-conditioned marginal conformal prediction methods for drug-target interaction prediction.<n>Group-conditioned CP works well when one entity is familiar, but residual-driven clustering provides robust uncertainty estimates even in sparse or novel scenarios.<n>These results highlight the potential of cluster-based CP for improving DTI prediction under uncertainty.
  arXiv  Detail & Related papers  (2025-05-24T22:31:49Z)
• Generalization and Informativeness of Conformal Prediction [36.407171992845456]
  Con conformal prediction (CP) transforms an arbitrary base predictor into a set predictor with coverage guarantees. CP certifies the predicted set to contain the target quantity with a user-defined tolerance, but it does not provide control over the average size of the predicted sets. A theoretical connection is established between the generalization properties of the base predictor and the informativeness of the resulting CP prediction sets. The derived upper bound provides insights into the dependence of the average size of the CP set predictor on the amount of calibration data, the target reliability, and the generalization performance of the base predictor.
  arXiv  Detail & Related papers  (2024-01-22T10:14:45Z)
• Efficient Conformal Prediction under Data Heterogeneity [79.35418041861327]
  Conformal Prediction (CP) stands out as a robust framework for uncertainty quantification. Existing approaches for tackling non-exchangeability lead to methods that are not computable beyond the simplest examples. This work introduces a new efficient approach to CP that produces provably valid confidence sets for fairly general non-exchangeable data distributions.
  arXiv  Detail & Related papers  (2023-12-25T20:02:51Z)
• Quantifying predictive uncertainty of aphasia severity in stroke patients with sparse heteroscedastic Bayesian high-dimensional regression [47.1405366895538]
  Sparse linear regression methods for high-dimensional data commonly assume that residuals have constant variance, which can be violated in practice. This paper proposes estimating high-dimensional heteroscedastic linear regression models using a heteroscedastic partitioned empirical Bayes Expectation Conditional Maximization algorithm.
  arXiv  Detail & Related papers  (2023-09-15T22:06:29Z)
• The Implicit Delta Method [61.36121543728134]
  In this paper, we propose an alternative, the implicit delta method, which works by infinitesimally regularizing the training loss of uncertainty. We show that the change in the evaluation due to regularization is consistent for the variance of the evaluation estimator, even when the infinitesimal change is approximated by a finite difference.
  arXiv  Detail & Related papers  (2022-11-11T19:34:17Z)
• Uncertainty Quantification for Traffic Forecasting: A Unified Approach [21.556559649467328]
  Uncertainty is an essential consideration for time series forecasting tasks. In this work, we focus on quantifying the uncertainty of traffic forecasting. We develop Deep S-Temporal Uncertainty Quantification (STUQ), which can estimate both aleatoric and relational uncertainty.
  arXiv  Detail & Related papers  (2022-08-11T15:21:53Z)
• A novel Deep Learning approach for one-step Conformal Prediction approximation [0.7646713951724009]
  Conformal Prediction (CP) is a versatile solution that guarantees a maximum error rate given minimal constraints. We propose a novel conformal loss function that approximates the traditionally two-step CP approach in a single step.
  arXiv  Detail & Related papers  (2022-07-25T17:46:09Z)
• Evaluating probabilistic classifiers: Reliability diagrams and score decompositions revisited [68.8204255655161]
  We introduce the CORP approach, which generates provably statistically Consistent, Optimally binned, and Reproducible reliability diagrams in an automated way. Corpor is based on non-parametric isotonic regression and implemented via the Pool-adjacent-violators (PAV) algorithm.
  arXiv  Detail & Related papers  (2020-08-07T08:22:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.