A computationally efficient neural network for predicting weather forecast probabilities

• URL: http://arxiv.org/abs/2103.14430v1
• Date: Fri, 26 Mar 2021 12:28:15 GMT
• Title: A computationally efficient neural network for predicting weather forecast probabilities
• Authors: Mariana Clare and Omar Jamil and Cyril Morcrette
• Abstract summary: We take the novel approach of using a neural network to predict probability density functions rather than a single output value. This enables the calculation of both uncertainty and skill metrics for the neural network predictions. This approach is purely data-driven and the neural network is trained on the WeatherBench dataset.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The success of deep learning techniques over the last decades has opened up a new avenue of research for weather forecasting. Here, we take the novel approach of using a neural network to predict probability density functions rather than a single output value, thus producing a probabilistic weather forecast. This enables the calculation of both uncertainty and skill metrics for the neural network predictions, and overcomes the common difficulty of inferring uncertainty from these predictions. This approach is purely data-driven and the neural network is trained on the WeatherBench dataset (processed ERA5 data) to forecast geopotential and temperature 3 and 5 days ahead. An extensive data exploration leads to the identification of the most important input variables, which are also found to agree with physical reasoning, thereby validating our approach. In order to increase computational efficiency further, each neural network is trained on a small subset of these variables. The outputs are then combined through a stacked neural network, the first time such a technique has been applied to weather data. Our approach is found to be more accurate than some numerical weather prediction models and as accurate as more complex alternative neural networks, with the added benefit of providing key probabilistic information necessary for making informed weather forecasts.

Related papers

• Deep Neural Networks Tend To Extrapolate Predictably [51.303814412294514]
  neural network predictions tend to be unpredictable and overconfident when faced with out-of-distribution (OOD) inputs. We observe that neural network predictions often tend towards a constant value as input data becomes increasingly OOD. We show how one can leverage our insights in practice to enable risk-sensitive decision-making in the presence of OOD inputs.
  arXiv  Detail & Related papers  (2023-10-02T03:25:32Z)
• Long-term drought prediction using deep neural networks based on geospatial weather data [75.38539438000072]
  High-quality drought forecasting up to a year in advance is critical for agriculture planning and insurance. We tackle drought data by introducing an end-to-end approach that adopts a systematic end-to-end approach. Key findings are the exceptional performance of a Transformer model, EarthFormer, in making accurate short-term (up to six months) forecasts.
  arXiv  Detail & Related papers  (2023-09-12T13:28:06Z)
• Deep Learning for Day Forecasts from Sparse Observations [60.041805328514876]
  Deep neural networks offer an alternative paradigm for modeling weather conditions. MetNet-3 learns from both dense and sparse data sensors and makes predictions up to 24 hours ahead for precipitation, wind, temperature and dew point. MetNet-3 has a high temporal and spatial resolution, respectively, up to 2 minutes and 1 km as well as a low operational latency.
  arXiv  Detail & Related papers  (2023-06-06T07:07:54Z)
• Semantic Strengthening of Neuro-Symbolic Learning [85.6195120593625]
  Neuro-symbolic approaches typically resort to fuzzy approximations of a probabilistic objective. We show how to compute this efficiently for tractable circuits. We test our approach on three tasks: predicting a minimum-cost path in Warcraft, predicting a minimum-cost perfect matching, and solving Sudoku puzzles.
  arXiv  Detail & Related papers  (2023-02-28T00:04:22Z)
• Confidence-Nets: A Step Towards better Prediction Intervals for regression Neural Networks on small datasets [0.0]
  We propose an ensemble method that attempts to estimate the uncertainty of predictions, increase their accuracy and provide an interval for the expected variation. The proposed method is tested on various datasets, and a significant improvement in the performance of the neural network model is seen.
  arXiv  Detail & Related papers  (2022-10-31T06:38:40Z)
• Scalable computation of prediction intervals for neural networks via matrix sketching [79.44177623781043]
  Existing algorithms for uncertainty estimation require modifying the model architecture and training procedure. This work proposes a new algorithm that can be applied to a given trained neural network and produces approximate prediction intervals.
  arXiv  Detail & Related papers  (2022-05-06T13:18:31Z)
• Short-term precipitation prediction using deep learning [5.1589108738893215]
  We show that a 3D convolutional neural network using a single frame of meteorology fields is capable of predicting the precipitation spatial distribution. The network is developed based on 39-years (1980-2018) data of meteorology and daily precipitation over the contiguous United States.
  arXiv  Detail & Related papers  (2021-10-05T06:37:24Z)
• Improving Uncertainty Calibration via Prior Augmented Data [56.88185136509654]
  Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators. They are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions. We propose a solution by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels.
  arXiv  Detail & Related papers  (2021-02-22T07:02:37Z)
• Accuracy of neural networks for the simulation of chaotic dynamics: precision of training data vs precision of the algorithm [0.0]
  We simulate the Lorenz system with different precisions using three different neural network techniques adapted to time series. Our results show that the ESN network is better at predicting accurately the dynamics of the system.
  arXiv  Detail & Related papers  (2020-07-08T17:25:37Z)
• Statistical post-processing of wind speed forecasts using convolutional neural networks [0.7646713951724009]
  We use spatial wind speed information by using convolutional neural networks (CNNs) and obtain probabilistic wind speed forecasts in the Netherlands for 48 hours ahead. The CNNs are shown to have higher Brier skill scores for medium to higher wind speeds, as well as a better continuous ranked probability score (CRPS) and logarithmic score, than the forecasts from fully connected neural networks and quantile regression forests.
  arXiv  Detail & Related papers  (2020-07-08T10:18:07Z)
• Ensemble methods for neural network-based weather forecasts [0.0]
  We aim to transform a deterministic neural network weather forecasting system into an ensemble forecasting system. We test four methods to generate the ensemble: random initial perturbations, retraining of the neural network, use of random dropout in the network, and the creation of initial perturbations with singular vector decomposition.
  arXiv  Detail & Related papers  (2020-02-13T09:28:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.