W-MAE: Pre-trained weather model with masked autoencoder for
multi-variable weather forecasting
- URL: http://arxiv.org/abs/2304.08754v2
- Date: Fri, 15 Dec 2023 17:34:36 GMT
- Title: W-MAE: Pre-trained weather model with masked autoencoder for
multi-variable weather forecasting
- Authors: Xin Man, Chenghong Zhang, Jin Feng, Changyu Li, Jie Shao
- Abstract summary: We propose a Weather model with Masked AutoEncoder pre-training for weather forecasting.
W-MAE is pre-trained in a self-supervised manner to reconstruct spatial correlations within meteorological variables.
On the temporal scale, we fine-tune the pre-trained W-MAE to predict the future states of meteorological variables.
- Score: 7.610811907813171
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Weather forecasting is a long-standing computational challenge with direct
societal and economic impacts. This task involves a large amount of continuous
data collection and exhibits rich spatiotemporal dependencies over long
periods, making it highly suitable for deep learning models. In this paper, we
apply pre-training techniques to weather forecasting and propose W-MAE, a
Weather model with Masked AutoEncoder pre-training for weather forecasting.
W-MAE is pre-trained in a self-supervised manner to reconstruct spatial
correlations within meteorological variables. On the temporal scale, we
fine-tune the pre-trained W-MAE to predict the future states of meteorological
variables, thereby modeling the temporal dependencies present in weather data.
We conduct our experiments using the fifth-generation ECMWF Reanalysis (ERA5)
data, with samples selected every six hours. Experimental results show that our
W-MAE framework offers three key benefits: 1) when predicting the future state
of meteorological variables, the utilization of our pre-trained W-MAE can
effectively alleviate the problem of cumulative errors in prediction,
maintaining stable performance in the short-to-medium term; 2) when predicting
diagnostic variables (e.g., total precipitation), our model exhibits
significant performance advantages over FourCastNet; 3) Our task-agnostic
pre-training schema can be easily integrated with various task-specific models.
When our pre-training framework is applied to FourCastNet, it yields an average
20% performance improvement in Anomaly Correlation Coefficient (ACC).
Related papers
- Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling [55.13352174687475]
This paper proposes a physics-AI hybrid model (i.e., WeatherGFT) which Generalizes weather forecasts to Finer-grained Temporal scales.
Specifically, we employ a carefully designed PDE kernel to simulate physical evolution on a small time scale.
We introduce a lead time-aware training framework to promote the generalization of the model at different lead times.
arXiv Detail & Related papers (2024-05-22T16:21:02Z) - Data Assimilation with Machine Learning Surrogate Models: A Case Study with FourCastNet [10.773673764125439]
This paper investigates online weather prediction using machine learning surrogates supplemented with partial and noisy observations.
We empirically demonstrate and theoretically justify that, despite the long-time instability of the surrogates, filtering estimates can remain accurate in the long-time horizon.
arXiv Detail & Related papers (2024-05-21T20:06:12Z) - EWMoE: An effective model for global weather forecasting with mixture-of-experts [6.695845790670147]
We propose EWMoE, an effective model for accurate global weather forecasting, which requires significantly less training data and computational resources.
Our model incorporates three key components to enhance prediction accuracy: meteorology-specific embedding, a core Mixture-of-Experts layer, and two specific loss functions.
arXiv Detail & Related papers (2024-05-09T16:42:13Z) - An ensemble of data-driven weather prediction models for operational sub-seasonal forecasting [0.08106028186803123]
We present an operations-ready multi-model ensemble weather forecasting system.
It is possible to achieve near-state-of-the-art subseasonal-to-seasonal forecasts using a multi-model ensembling approach with data-driven weather prediction models.
arXiv Detail & Related papers (2024-03-22T20:01:53Z) - Uncertainty quantification for data-driven weather models [0.0]
We study and compare uncertainty quantification methods to generate probabilistic weather forecasts from a state-of-the-art deterministic data-driven weather model, Pangu-Weather.
Specifically, we compare approaches for quantifying forecast uncertainty based on generating ensemble forecasts via perturbations to the initial conditions.
In a case study on medium-range forecasts of selected weather variables over Europe, the probabilistic forecasts obtained by using the Pangu-Weather model in concert with uncertainty quantification methods show promising results.
arXiv Detail & Related papers (2024-03-20T10:07:51Z) - ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast.
We also introduce a training-free extreme value enhancement strategy named ExEnsemble, which increases the variance of pixel values and improves the forecast robustness.
Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
arXiv Detail & Related papers (2024-02-02T10:34:13Z) - FengWu-4DVar: Coupling the Data-driven Weather Forecasting Model with 4D Variational Assimilation [67.20588721130623]
We develop an AI-based cyclic weather forecasting system, FengWu-4DVar.
FengWu-4DVar can incorporate observational data into the data-driven weather forecasting model.
Experiments on the simulated observational dataset demonstrate that FengWu-4DVar is capable of generating reasonable analysis fields.
arXiv Detail & Related papers (2023-12-16T02:07:56Z) - Long-term drought prediction using deep neural networks based on geospatial weather data [75.38539438000072]
High-quality drought forecasting up to a year in advance is critical for agriculture planning and insurance.
We tackle drought data by introducing an end-to-end approach that adopts a systematic end-to-end approach.
Key findings are the exceptional performance of a Transformer model, EarthFormer, in making accurate short-term (up to six months) forecasts.
arXiv Detail & Related papers (2023-09-12T13:28:06Z) - Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global
Weather Forecast [91.9372563527801]
We present Pangu-Weather, a deep learning based system for fast and accurate global weather forecast.
For the first time, an AI-based method outperforms state-of-the-art numerical weather prediction (NWP) methods in terms of accuracy.
Pangu-Weather supports a wide range of downstream forecast scenarios, including extreme weather forecast and large-member ensemble forecast in real-time.
arXiv Detail & Related papers (2022-11-03T17:19:43Z) - A generative adversarial network approach to (ensemble) weather
prediction [91.3755431537592]
We use a conditional deep convolutional generative adversarial network to predict the geopotential height of the 500 hPa pressure level, the two-meter temperature and the total precipitation for the next 24 hours over Europe.
The proposed models are trained on 4 years of ERA5 reanalysis data from 2015-2018 with the goal to predict the associated meteorological fields in 2019.
arXiv Detail & Related papers (2020-06-13T20:53:17Z) - A framework for probabilistic weather forecast post-processing across
models and lead times using machine learning [3.1542695050861544]
We show how to bridge the gap between sets of separate forecasts from NWP models and the 'ideal' forecast for decision support.
We use Quantile Regression Forests to learn the error profile of each numerical model, and use these to apply empirically-derived probability distributions to forecasts.
Second, we combine these probabilistic forecasts using quantile averaging. Third, we interpolate between the aggregate quantiles in order to generate a full predictive distribution.
arXiv Detail & Related papers (2020-05-06T16:46:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.