Computing the ensemble spread from deterministic weather predictions using conditional generative adversarial networks

• URL: http://arxiv.org/abs/2205.09182v1
• Date: Wed, 18 May 2022 19:10:38 GMT
• Title: Computing the ensemble spread from deterministic weather predictions using conditional generative adversarial networks
• Authors: R\"udiger Brecht and Alex Bihlo
• Abstract summary: We propose to use deep-learning algorithms to learn the statistical properties of an ensemble prediction system. Once trained, the costly ensemble prediction system will not be needed anymore to obtain future ensemble forecasts.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ensemble prediction systems are an invaluable tool for weather forecasting. Practically, ensemble predictions are obtained by running several perturbations of the deterministic control forecast. However, ensemble prediction is associated with a high computational cost and often involves statistical post-processing steps to improve its quality. Here we propose to use deep-learning-based algorithms to learn the statistical properties of an ensemble prediction system, the ensemble spread, given only the deterministic control forecast. Thus, once trained, the costly ensemble prediction system will not be needed anymore to obtain future ensemble forecasts, and the statistical properties of the ensemble can be derived from a single deterministic forecast. We adapt the classical pix2pix architecture to a three-dimensional model and also experiment with a shared latent space encoder-decoder model, and train them against several years of operational (ensemble) weather forecasts for the 500 hPa geopotential height. The results demonstrate that the trained models indeed allow obtaining a highly accurate ensemble spread from the control forecast only.

Related papers

• Probabilistic Weather Forecasting with Hierarchical Graph Neural Networks [17.64833210797824]
  We propose a probabilistic weather forecasting model called Graph-EFM. The model combines a flexible latent-variable formulation with the successful graph-based forecasting framework. Ensemble forecasts from Graph-EFM achieve equivalent or lower errors than comparable deterministic models.
  arXiv  Detail & Related papers  (2024-06-07T09:01:25Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2023-10-17T20:30:16Z)
• SEEDS: Emulation of Weather Forecast Ensembles with Diffusion Models [13.331224394143117]
  Uncertainty quantification is crucial to decision-making. dominant approach to representing uncertainty in weather forecasting is to generate an ensemble of forecasts. We propose to amortize the computational cost by emulating these forecasts with deep generative diffusion models learned from historical data.
  arXiv  Detail & Related papers  (2023-06-24T22:00:06Z)
• Prediction-Powered Inference [68.97619568620709]
  Prediction-powered inference is a framework for performing valid statistical inference when an experimental dataset is supplemented with predictions from a machine-learning system. The framework yields simple algorithms for computing provably valid confidence intervals for quantities such as means, quantiles, and linear and logistic regression coefficients. Prediction-powered inference could enable researchers to draw valid and more data-efficient conclusions using machine learning.
  arXiv  Detail & Related papers  (2023-01-23T18:59:28Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2022-06-16T06:13:53Z)
• Long Range Probabilistic Forecasting in Time-Series using High Order Statistics [19.12411040726229]
  We present a novel probabilistic forecasting method that produces forecasts coherent in terms of base level and predicted aggregate statistics. We show that our method improves forecast performance across both base level and unseen aggregates post inference on real datasets ranging three diverse domains.
  arXiv  Detail & Related papers  (2021-11-05T11:10:35Z)
• Test-time Collective Prediction [73.74982509510961]
  Multiple parties in machine learning want to jointly make predictions on future test points. Agents wish to benefit from the collective expertise of the full set of agents, but may not be willing to release their data or model parameters. We explore a decentralized mechanism to make collective predictions at test time, leveraging each agent's pre-trained model.
  arXiv  Detail & Related papers  (2021-06-22T18:29:58Z)
• A generative adversarial network approach to (ensemble) weather prediction [91.3755431537592]
  We use a conditional deep convolutional generative adversarial network to predict the geopotential height of the 500 hPa pressure level, the two-meter temperature and the total precipitation for the next 24 hours over Europe. The proposed models are trained on 4 years of ERA5 reanalysis data from 2015-2018 with the goal to predict the associated meteorological fields in 2019.
  arXiv  Detail & Related papers  (2020-06-13T20:53:17Z)
• A framework for probabilistic weather forecast post-processing across models and lead times using machine learning [3.1542695050861544]
  We show how to bridge the gap between sets of separate forecasts from NWP models and the 'ideal' forecast for decision support. We use Quantile Regression Forests to learn the error profile of each numerical model, and use these to apply empirically-derived probability distributions to forecasts. Second, we combine these probabilistic forecasts using quantile averaging. Third, we interpolate between the aggregate quantiles in order to generate a full predictive distribution.
  arXiv  Detail & Related papers  (2020-05-06T16:46:02Z)
• Ensemble methods for neural network-based weather forecasts [0.0]
  We aim to transform a deterministic neural network weather forecasting system into an ensemble forecasting system. We test four methods to generate the ensemble: random initial perturbations, retraining of the neural network, use of random dropout in the network, and the creation of initial perturbations with singular vector decomposition.
  arXiv  Detail & Related papers  (2020-02-13T09:28:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.