Deep Learning for Post-Processing Ensemble Weather Forecasts

• URL: http://arxiv.org/abs/2005.08748v2
• Date: Mon, 21 Sep 2020 11:08:47 GMT
• Title: Deep Learning for Post-Processing Ensemble Weather Forecasts
• Authors: Peter Gr\"onquist, Chengyuan Yao, Tal Ben-Nun, Nikoli Dryden, Peter Dueben, Shigang Li, Torsten Hoefler
• Abstract summary: We propose a mixed model that uses only a subset of the original weather trajectories combined with a post-processing step using deep neural networks. We show that our post-processing can use fewer trajectories to achieve comparable results to the full ensemble.
• Score: 14.622977874836298
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quantifying uncertainty in weather forecasts is critical, especially for predicting extreme weather events. This is typically accomplished with ensemble prediction systems, which consist of many perturbed numerical weather simulations, or trajectories, run in parallel. These systems are associated with a high computational cost and often involve statistical post-processing steps to inexpensively improve their raw prediction qualities. We propose a mixed model that uses only a subset of the original weather trajectories combined with a post-processing step using deep neural networks. These enable the model to account for non-linear relationships that are not captured by current numerical models or post-processing methods. Applied to global data, our mixed models achieve a relative improvement in ensemble forecast skill (CRPS) of over 14%. Furthermore, we demonstrate that the improvement is larger for extreme weather events on select case studies. We also show that our post-processing can use fewer trajectories to achieve comparable results to the full ensemble. By using fewer trajectories, the computational costs of an ensemble prediction system can be reduced, allowing it to run at higher resolution and produce more accurate forecasts.

Related papers

• PostCast: Generalizable Postprocessing for Precipitation Nowcasting via Unsupervised Blurriness Modeling [85.56969895866243]
  We propose an unsupervised postprocessing method to eliminate the blurriness without the requirement of training with the pairs of blurry predictions and corresponding ground truth. A zero-shot blur kernel estimation mechanism and an auto-scale denoise guidance strategy are introduced to adapt the unconditional correlations to any blurriness modes.
  arXiv  Detail & Related papers  (2024-10-08T08:38:23Z)
• Continuous Ensemble Weather Forecasting with Diffusion models [10.730406954385927]
  Continuous Ensemble Forecasting is a novel and flexible method for sampling ensemble forecasts in diffusion models. It can generate temporally consistent ensemble trajectories completely in parallel, with no autoregressive steps. We demonstrate that the method achieves competitive results for global weather forecasting with good probabilistic properties.
  arXiv  Detail & Related papers  (2024-10-07T18:51:23Z)
• Forecast-PEFT: Parameter-Efficient Fine-Tuning for Pre-trained Motion Forecasting Models [68.23649978697027]
  Forecast-PEFT is a fine-tuning strategy that freezes the majority of the model's parameters, focusing adjustments on newly introduced prompts and adapters. Our experiments show that Forecast-PEFT outperforms traditional full fine-tuning methods in motion prediction tasks. Forecast-FT further improves prediction performance, evidencing up to a 9.6% enhancement over conventional baseline methods.
  arXiv  Detail & Related papers  (2024-07-28T19:18:59Z)
• Improving probabilistic forecasts of extreme wind speeds by training statistical post-processing models with weighted scoring rules [0.0]
  Training using the threshold-weighted continuous ranked probability score (twCRPS) leads to improved extreme event performance of post-processing models. We find a distribution body-tail trade-off where improved performance for probabilistic predictions of extreme events comes with worse performance for predictions of the distribution body.
  arXiv  Detail & Related papers  (2024-07-22T11:07:52Z)
• Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling [55.13352174687475]
  This paper proposes a physics-AI hybrid model (i.e., WeatherGFT) which Generalizes weather forecasts to Finer-grained Temporal scales. Specifically, we employ a carefully designed PDE kernel to simulate physical evolution on a small time scale. We introduce a lead time-aware training framework to promote the generalization of the model at different lead times.
  arXiv  Detail & Related papers  (2024-05-22T16:21:02Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• SEEDS: Emulation of Weather Forecast Ensembles with Diffusion Models [13.331224394143117]
  Uncertainty quantification is crucial to decision-making. dominant approach to representing uncertainty in weather forecasting is to generate an ensemble of forecasts. We propose to amortize the computational cost by emulating these forecasts with deep generative diffusion models learned from historical data.
  arXiv  Detail & Related papers  (2023-06-24T22:00:06Z)
• Forecast reconciliation for vaccine supply chain optimization [61.13962963550403]
  Vaccine supply chain optimization can benefit from hierarchical time series forecasting. Forecasts of different hierarchy levels become incoherent when higher levels do not match the sum of the lower levels forecasts. We tackle the vaccine sale forecasting problem by modeling sales data from GSK between 2010 and 2021 as a hierarchical time series.
  arXiv  Detail & Related papers  (2023-05-02T14:34:34Z)
• Towards replacing precipitation ensemble predictions systems using machine learning [0.0]
  We propose a new approach to generating ensemble weather predictions for high-resolution precipitation. The method uses generative adversarial networks to learn the complex patterns of precipitation. We demonstrate the feasibility of generating realistic precipitation ensemble members on unseen higher resolutions.
  arXiv  Detail & Related papers  (2023-04-20T12:20:35Z)
• Machine learning methods for postprocessing ensemble forecasts of wind gusts: A systematic comparison [0.0]
  Postprocessing ensemble weather predictions to correct systematic errors has become a standard practice in research and operations. Here, we provide a comprehensive review and systematic comparison of eight statistical and machine learning methods for probabilistic wind gust forecasting. We propose a flexible framework of locally adaptive neural networks with different probabilistic forecast types as output.
  arXiv  Detail & Related papers  (2021-06-17T14:03:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.