EWMoE: An effective model for global weather forecasting with mixture-of-experts
- URL: http://arxiv.org/abs/2405.06004v1
- Date: Thu, 9 May 2024 16:42:13 GMT
- Title: EWMoE: An effective model for global weather forecasting with mixture-of-experts
- Authors: Lihao Gan, Xin Man, Chenghong Zhang, Jie Shao,
- Abstract summary: We propose EWMoE, an effective model for accurate global weather forecasting, which requires significantly less training data and computational resources.
Our model incorporates three key components to enhance prediction accuracy: meteorology-specific embedding, a core Mixture-of-Experts layer, and two specific loss functions.
- Score: 6.695845790670147
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Weather forecasting is a crucial task for meteorologic research, with direct social and economic impacts. Recently, data-driven weather forecasting models based on deep learning have shown great potential, achieving superior performance compared with traditional numerical weather prediction methods. However, these models often require massive training data and computational resources. In this paper, we propose EWMoE, an effective model for accurate global weather forecasting, which requires significantly less training data and computational resources. Our model incorporates three key components to enhance prediction accuracy: meteorology-specific embedding, a core Mixture-of-Experts (MoE) layer, and two specific loss functions. We conduct our evaluation on the ERA5 dataset using only two years of training data. Extensive experiments demonstrate that EWMoE outperforms current models such as FourCastNet and ClimaX at all forecast time, achieving competitive performance compared with the state-of-the-art Pangu-Weather model in evaluation metrics such as Anomaly Correlation Coefficient (ACC) and Root Mean Square Error (RMSE). Additionally, ablation studies indicate that applying the MoE architecture to weather forecasting offers significant advantages in improving accuracy and resource efficiency.
Related papers
- Validating Deep-Learning Weather Forecast Models on Recent High-Impact Extreme Events [0.1747623282473278]
We compare weather prediction models and ECMWF's high-resolution forecast (HRES) system in three case studies.
We find evidence that machine learning weather prediction models can achieve similar accuracy to HRES on record-shattering events.
However, extrapolating to extreme conditions may impact machine learning models more severely than HRES.
arXiv Detail & Related papers (2024-04-26T18:18:25Z) - Uncertainty quantification for data-driven weather models [0.0]
We study and compare uncertainty quantification methods to generate probabilistic weather forecasts from a state-of-the-art deterministic data-driven weather model, Pangu-Weather.
Specifically, we compare approaches for quantifying forecast uncertainty based on generating ensemble forecasts via perturbations to the initial conditions.
In a case study on medium-range forecasts of selected weather variables over Europe, the probabilistic forecasts obtained by using the Pangu-Weather model in concert with uncertainty quantification methods show promising results.
arXiv Detail & Related papers (2024-03-20T10:07:51Z) - Advancing Data-driven Weather Forecasting: Time-Sliding Data
Augmentation of ERA5 [3.3748750222488657]
We introduce a novel strategy that deviates from the common dependence on high-resolution data.
This paper improves on conventional approaches by adding more variables and a novel approach to data augmentation and processing.
Our findings reveal that despite the lower resolution, the proposed approach demonstrates considerable accuracy in predicting atmospheric conditions.
arXiv Detail & Related papers (2024-02-13T03:01:22Z) - ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast.
We also introduce a training-free extreme value enhancement strategy named ExEnsemble, which increases the variance of pixel values and improves the forecast robustness.
Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
arXiv Detail & Related papers (2024-02-02T10:34:13Z) - FengWu-GHR: Learning the Kilometer-scale Medium-range Global Weather
Forecasting [56.73502043159699]
This work presents FengWu-GHR, the first data-driven global weather forecasting model running at the 0.09$circ$ horizontal resolution.
It introduces a novel approach that opens the door for operating ML-based high-resolution forecasts by inheriting prior knowledge from a low-resolution model.
The hindcast of weather prediction in 2022 indicates that FengWu-GHR is superior to the IFS-HRES.
arXiv Detail & Related papers (2024-01-28T13:23:25Z) - FengWu-4DVar: Coupling the Data-driven Weather Forecasting Model with 4D Variational Assimilation [67.20588721130623]
We develop an AI-based cyclic weather forecasting system, FengWu-4DVar.
FengWu-4DVar can incorporate observational data into the data-driven weather forecasting model.
Experiments on the simulated observational dataset demonstrate that FengWu-4DVar is capable of generating reasonable analysis fields.
arXiv Detail & Related papers (2023-12-16T02:07:56Z) - Long-term drought prediction using deep neural networks based on geospatial weather data [75.38539438000072]
High-quality drought forecasting up to a year in advance is critical for agriculture planning and insurance.
We tackle drought data by introducing an end-to-end approach that adopts a systematic end-to-end approach.
Key findings are the exceptional performance of a Transformer model, EarthFormer, in making accurate short-term (up to six months) forecasts.
arXiv Detail & Related papers (2023-09-12T13:28:06Z) - W-MAE: Pre-trained weather model with masked autoencoder for
multi-variable weather forecasting [7.610811907813171]
We propose a Weather model with Masked AutoEncoder pre-training for weather forecasting.
W-MAE is pre-trained in a self-supervised manner to reconstruct spatial correlations within meteorological variables.
On the temporal scale, we fine-tune the pre-trained W-MAE to predict the future states of meteorological variables.
arXiv Detail & Related papers (2023-04-18T06:25:11Z) - GraphCast: Learning skillful medium-range global weather forecasting [107.40054095223779]
We introduce a machine learning-based method called "GraphCast", which can be trained directly from reanalysis data.
It predicts hundreds of weather variables, over 10 days at 0.25 degree resolution globally, in under one minute.
We show that GraphCast significantly outperforms the most accurate operational deterministic systems on 90% of 1380 verification targets.
arXiv Detail & Related papers (2022-12-24T18:15:39Z) - A generative adversarial network approach to (ensemble) weather
prediction [91.3755431537592]
We use a conditional deep convolutional generative adversarial network to predict the geopotential height of the 500 hPa pressure level, the two-meter temperature and the total precipitation for the next 24 hours over Europe.
The proposed models are trained on 4 years of ERA5 reanalysis data from 2015-2018 with the goal to predict the associated meteorological fields in 2019.
arXiv Detail & Related papers (2020-06-13T20:53:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.