Postprocessing of Ensemble Weather Forecasts Using Permutation-invariant Neural Networks

• URL: http://arxiv.org/abs/2309.04452v2
• Date: Thu, 18 Jan 2024 19:02:54 GMT
• Title: Postprocessing of Ensemble Weather Forecasts Using Permutation-invariant Neural Networks
• Authors: Kevin H\"ohlein, Benedikt Schulz, R\"udiger Westermann and Sebastian Lerch
• Abstract summary: We propose networks that treat forecast ensembles as a set of unordered member forecasts. We evaluate the quality of the obtained forecast distributions in terms of calibration and sharpness. Our results suggest that most of the relevant information is contained in a few ensemble-internal degrees of freedom.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Statistical postprocessing is used to translate ensembles of raw numerical weather forecasts into reliable probabilistic forecast distributions. In this study, we examine the use of permutation-invariant neural networks for this task. In contrast to previous approaches, which often operate on ensemble summary statistics and dismiss details of the ensemble distribution, we propose networks that treat forecast ensembles as a set of unordered member forecasts and learn link functions that are by design invariant to permutations of the member ordering. We evaluate the quality of the obtained forecast distributions in terms of calibration and sharpness and compare the models against classical and neural network-based benchmark methods. In case studies addressing the postprocessing of surface temperature and wind gust forecasts, we demonstrate state-of-the-art prediction quality. To deepen the understanding of the learned inference process, we further propose a permutation-based importance analysis for ensemble-valued predictors, which highlights specific aspects of the ensemble forecast that are considered important by the trained postprocessing models. Our results suggest that most of the relevant information is contained in a few ensemble-internal degrees of freedom, which may impact the design of future ensemble forecasting and postprocessing systems.

Related papers

• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2023-10-17T20:30:16Z)
• Forecast reconciliation for vaccine supply chain optimization [61.13962963550403]
  Vaccine supply chain optimization can benefit from hierarchical time series forecasting. Forecasts of different hierarchy levels become incoherent when higher levels do not match the sum of the lower levels forecasts. We tackle the vaccine sale forecasting problem by modeling sales data from GSK between 2010 and 2021 as a hierarchical time series.
  arXiv  Detail & Related papers  (2023-05-02T14:34:34Z)
• Ensemble weather forecast post-processing with a flexible probabilistic neural network approach [0.0]
  We propose a novel, neural network-based method, which produces forecasts for all locations and lead times jointly. We demonstrate the effectiveness of our method in the context of the EUPPBench benchmark.
  arXiv  Detail & Related papers  (2023-03-29T15:18:00Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2022-06-16T06:13:53Z)
• Computing the ensemble spread from deterministic weather predictions using conditional generative adversarial networks [0.0]
  We propose to use deep-learning algorithms to learn the statistical properties of an ensemble prediction system. Once trained, the costly ensemble prediction system will not be needed anymore to obtain future ensemble forecasts.
  arXiv  Detail & Related papers  (2022-05-18T19:10:38Z)
• Aggregating distribution forecasts from deep ensembles [0.0]
  We study the question of how to aggregate distribution forecasts based on neural network-based approaches. We show that combining forecast distributions can substantially improve the predictive performance. We propose a general quantile aggregation framework for deep ensembles that shows superior performance compared to a linear combination of the forecast densities.
  arXiv  Detail & Related papers  (2022-04-05T15:42:51Z)
• Distributional Gradient Boosting Machines [77.34726150561087]
  Our framework is based on XGBoost and LightGBM. We show that our framework achieves state-of-the-art forecast accuracy.
  arXiv  Detail & Related papers  (2022-04-02T06:32:19Z)
• Machine learning methods for postprocessing ensemble forecasts of wind gusts: A systematic comparison [0.0]
  Postprocessing ensemble weather predictions to correct systematic errors has become a standard practice in research and operations. Here, we provide a comprehensive review and systematic comparison of eight statistical and machine learning methods for probabilistic wind gust forecasting. We propose a flexible framework of locally adaptive neural networks with different probabilistic forecast types as output.
  arXiv  Detail & Related papers  (2021-06-17T14:03:29Z)
• Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting [98.57851612518758]
  Probabilistic time series forecasting involves estimating the distribution of future based on its history. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks. We show in experiments that our model produces accurate and sharp probabilistic forecasts.
  arXiv  Detail & Related papers  (2021-01-31T06:49:33Z)
• Probabilistic electric load forecasting through Bayesian Mixture Density Networks [70.50488907591463]
  Probabilistic load forecasting (PLF) is a key component in the extended tool-chain required for efficient management of smart energy grids. We propose a novel PLF approach, framed on Bayesian Mixture Density Networks. To achieve reliable and computationally scalable estimators of the posterior distributions, both Mean Field variational inference and deep ensembles are integrated.
  arXiv  Detail & Related papers  (2020-12-23T16:21:34Z)
• Forecasting time series with encoder-decoder neural networks [0.0]
  We consider high-dimensional stationary processes where a new observation is generated from a compressed version of past observations. The specific evolution is modeled by an encoder-decoder structure. We estimate the evolution with an encoder-decoder neural network and give upper bounds for the expected forecast error.
  arXiv  Detail & Related papers  (2020-09-18T14:07:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.