Reliability of Probabilistic Emulation of Physical Systems

• URL: http://arxiv.org/abs/2606.12997v1
• Date: Thu, 11 Jun 2026 07:34:41 GMT
• Title: Reliability of Probabilistic Emulation of Physical Systems
• Abstract summary: We develop a framework to evaluate generative models and CRPS-trained ensembles across diverse 2D physical systems.<n>Both generative models and CRPS-trained ensembles demonstrate good predictive accuracy.<n>To facilitate future research and application, we release AutoCast, a modular framework implementing both generative models and CRPS-trained ensembles.
• Score: 2.388494426638727
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Two dominant approaches have emerged for generating probabilistic forecasts of physical systems: generative models, such as diffusion or flow matching; and ensembles of deterministic models with stochasticity injected, trained using the continuous ranked probability score (CRPS) loss. While both approaches have demonstrated strong predictive accuracy, the reliability of their uncertainties has not been systematically assessed. We address this gap by developing a framework to evaluate both approaches across diverse 2D spatiotemporal physical systems, under matched model size and computational budget. We assess the reliability of probabilistic emulation by inspecting the empirical coverage of predictive intervals, while also considering accuracy and computational efficiency metrics. CRPS-trained ensembles typically achieve more reliable uncertainties on both single-step prediction and autoregressive rollouts, demonstrating better coverage than the standard alternative of training generative models in a latent space. Moreover, the CRPS approach offers significantly faster inference. When generative models are trained in ambient rather than a compressed latent space, which is often infeasible for high-dimensional problems, they exhibit comparable coverage to CRPS-trained ensembles, though with substantially larger inference latency. In contrast, when CRPS-trained ensembles are trained in latent space they do not show a marked degradation in coverage with respect to ambient space. Both generative models and CRPS-trained ensembles demonstrate good predictive accuracy. To facilitate future research and application, we release AutoCast, a modular framework implementing both generative models and CRPS-trained ensembles, alongside AutoSim, a flexible dataset generation package for rapid prototyping.

Related papers

• Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy [51.758416625168]
  Randomized smoothing is a powerful tool for certifying robustness to adversarial perturbations.<n>We propose a framework for certification of complex, composed mechanisms.<n>We provide a principled and general framework for using composite mechanisms to certify robustness under complex threat models.
  arXiv  Detail & Related papers  (2026-05-20T22:17:29Z)
• Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration [72.0672328514289]
  Multimodal learning often grapples with the challenge of low-quality data.<n>We propose a unified framework, termed Conformal Predictive Self-Calibration.<n>Our framework consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-05T14:48:52Z)
• A Hybrid Intelligent Framework for Uncertainty-Aware Condition Monitoring of Industrial Systems [4.854320235705813]
  This work develops a hybrid condition monitoring framework that integrates primary sensor measurements, lagged temporal features, and physics-informed residuals.<n>Two hybrid integration strategies are examined. The first is a feature-level fusion approach that augments the input space with residual and temporal information.<n>The second is a model-level ensemble approach in which machine learning classifiers trained on different feature types are combined at the decision level.
  arXiv  Detail & Related papers  (2026-04-10T22:22:31Z)
• Demystifying Data-Driven Probabilistic Medium-Range Weather Forecasting [63.8116386935854]
  We demonstrate that state-of-the-art probabilistic skill requires neither intricate architectural constraints nor specialized trainings.<n>We introduce a scalable framework for learning multi-scale atmospheric dynamics by combining a directly downsampled latent space with a history-conditioned local projector.<n>We find that our framework design is robust to the choice of probabilistic estimators, seamlessly supporting interpolants, diffusion models, and CRPS-based ensemble training.
  arXiv  Detail & Related papers  (2026-01-26T03:52:16Z)
• Conformal prediction for full and sparse polynomial chaos expansions [0.5735035463793009]
  Polynomial Chaos Expansions (PCEs) are widely recognized for their efficient computational performance in surrogate modeling.<n>We present the integration of two conformal prediction methods, namely the full conformal and the Jackknife+ approaches, into both full and sparse PCEs.
  arXiv  Detail & Related papers  (2026-01-23T10:46:46Z)
• Bridging the Gap Between Bayesian Deep Learning and Ensemble Weather Forecasts [100.26854618129039]
  Weather forecasting is fundamentally challenged by the chaotic nature of the atmosphere.<n>Recent advances in Bayesian Deep Learning (BDL) offer a promising but often disconnected alternative.<n>We bridge these paradigms through a unified hybrid BDL framework for ensemble weather forecasting.
  arXiv  Detail & Related papers  (2025-11-18T07:49:52Z)
• FUSE: Fast Unified Simulation and Estimation for PDEs [11.991297011923004]
  We argue that solving both problems within the same framework can lead to consistent gains in accuracy and robustness. We present the capabilities of the proposed methodology for predicting continuous and discrete biomarkers in full-body haemodynamics simulations.
  arXiv  Detail & Related papers  (2024-05-23T13:37:26Z)
• Probabilistic electric load forecasting through Bayesian Mixture Density Networks [70.50488907591463]
  Probabilistic load forecasting (PLF) is a key component in the extended tool-chain required for efficient management of smart energy grids. We propose a novel PLF approach, framed on Bayesian Mixture Density Networks. To achieve reliable and computationally scalable estimators of the posterior distributions, both Mean Field variational inference and deep ensembles are integrated.
  arXiv  Detail & Related papers  (2020-12-23T16:21:34Z)
• At the Intersection of Deep Sequential Model Framework and State-space Model Framework: Study on Option Pricing [2.3224617218247126]
  Inference and forecast problems of the nonlinear dynamical system have arisen in a variety of contexts. We propose a model that unifies both deep sequential and state-space models to achieve both frameworks' superiorities.
  arXiv  Detail & Related papers  (2020-12-14T18:21:41Z)
• Trust but Verify: Assigning Prediction Credibility by Counterfactual Constrained Learning [123.3472310767721]
  Prediction credibility measures are fundamental in statistics and machine learning. These measures should account for the wide variety of models used in practice. The framework developed in this work expresses the credibility as a risk-fit trade-off.
  arXiv  Detail & Related papers  (2020-11-24T19:52:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.