Exploring Design Choices for Autoregressive Deep Learning Climate Models
- URL: http://arxiv.org/abs/2505.02506v1
- Date: Mon, 05 May 2025 09:37:58 GMT
- Title: Exploring Design Choices for Autoregressive Deep Learning Climate Models
- Authors: Florian Gallusser, Simon Hentschel, Anna Krause, Andreas Hotho,
- Abstract summary: This study quantitatively compares the long-term stability of three prominent DL-MWP architectures trained on ERA5 reanalysis data at 5.625deg resolution.<n>We identify configurations that enable stable 10-year rollouts while preserving the statistical properties of the reference dataset.
- Score: 2.401696775092447
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Deep Learning models have achieved state-of-the-art performance in medium-range weather prediction but often fail to maintain physically consistent rollouts beyond 14 days. In contrast, a few atmospheric models demonstrate stability over decades, though the key design choices enabling this remain unclear. This study quantitatively compares the long-term stability of three prominent DL-MWP architectures - FourCastNet, SFNO, and ClimaX - trained on ERA5 reanalysis data at 5.625{\deg} resolution. We systematically assess the impact of autoregressive training steps, model capacity, and choice of prognostic variables, identifying configurations that enable stable 10-year rollouts while preserving the statistical properties of the reference dataset. Notably, rollouts with SFNO exhibit the greatest robustness to hyperparameter choices, yet all models can experience instability depending on the random seed and the set of prognostic variables
Related papers
- AtmosMJ: Revisiting Gating Mechanism for AI Weather Forecasting Beyond the Year Scale [4.8951183832371]
We introduce a deep convolutional network that operates directly on ERA5 data without any spherical remapping.<n>Our results demonstrate that AtmosMJ produces stable and physically plausible forecasts for about 500 days.
arXiv Detail & Related papers (2025-06-11T13:38:56Z) - Weakly-Constrained 4D Var for Downscaling with Uncertainty using Data-Driven Surrogate Models [1.3654846342364308]
Dynamic downscaling typically involves using numerical weather prediction solvers to refine coarse data to higher spatial resolutions.<n>Data-driven models such as FourCastNet have emerged as a promising alternative to the traditional NWP models for forecasting.<n>We propose to use data assimilation approaches to stabilize them when used for downscaling tasks.
arXiv Detail & Related papers (2025-03-04T14:33:54Z) - Skillful High-Resolution Ensemble Precipitation Forecasting with an Integrated Deep Learning Framework [4.3313006430322165]
High-resolution precipitation forecasts are crucial for providing accurate weather prediction and supporting effective responses to extreme weather events.<n>We propose a physics-inspired deep learning framework for high-resolution ensemble precipitation forecasting.
arXiv Detail & Related papers (2025-01-06T10:29:38Z) - On conditional diffusion models for PDE simulations [53.01911265639582]
We study score-based diffusion models for forecasting and assimilation of sparse observations.
We propose an autoregressive sampling approach that significantly improves performance in forecasting.
We also propose a new training strategy for conditional score-based models that achieves stable performance over a range of history lengths.
arXiv Detail & Related papers (2024-10-21T18:31:04Z) - Weather Prediction with Diffusion Guided by Realistic Forecast Processes [49.07556359513563]
We introduce a novel method that applies diffusion models (DM) for weather forecasting.
Our method can achieve both direct and iterative forecasting with the same modeling framework.
The flexibility and controllability of our model empowers a more trustworthy DL system for the general weather community.
arXiv Detail & Related papers (2024-02-06T21:28:42Z) - FengWu-4DVar: Coupling the Data-driven Weather Forecasting Model with 4D Variational Assimilation [67.20588721130623]
We develop an AI-based cyclic weather forecasting system, FengWu-4DVar.
FengWu-4DVar can incorporate observational data into the data-driven weather forecasting model.
Experiments on the simulated observational dataset demonstrate that FengWu-4DVar is capable of generating reasonable analysis fields.
arXiv Detail & Related papers (2023-12-16T02:07:56Z) - Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation [67.18144414660681]
We propose a Fast-Slow Test-Time Adaptation (FSTTA) approach for online Vision-and-Language Navigation (VLN)
Our method obtains impressive performance gains on four popular benchmarks.
arXiv Detail & Related papers (2023-11-22T07:47:39Z) - Long-term drought prediction using deep neural networks based on geospatial weather data [75.38539438000072]
High-quality drought forecasting up to a year in advance is critical for agriculture planning and insurance.
We tackle drought data by introducing an end-to-end approach that adopts a systematic end-to-end approach.
Key findings are the exceptional performance of a Transformer model, EarthFormer, in making accurate short-term (up to six months) forecasts.
arXiv Detail & Related papers (2023-09-12T13:28:06Z) - Challenges of learning multi-scale dynamics with AI weather models: Implications for stability and one solution [0.0]
Current AI-based weather models can only provide short-term forecasts accurately when time-integrated beyond a few weeks or a few months.<n>The cause of the instabilities is unknown, and the methods that are used to improve their stability horizons are ad-hoc and lack rigorous theory.<n>We develop long-term physically-consistent data-driven models for the climate system and demonstrate accurate short-term forecasts.
arXiv Detail & Related papers (2023-04-14T09:49:11Z) - Exploring The Landscape of Distributional Robustness for Question
Answering Models [47.178481044045505]
Investigation spans over 350 models and 16 question answering datasets.
We find that, in many cases, model variations do not affect robustness.
We release all evaluations to encourage researchers to further analyze robustness trends for question answering models.
arXiv Detail & Related papers (2022-10-22T18:17:31Z) - Long-term stability and generalization of observationally-constrained
stochastic data-driven models for geophysical turbulence [0.19686770963118383]
Deep learning models can mitigate certain biases in current state-of-the-art weather models.
Data-driven models require a lot of training data which may not be available from reanalysis (observational data) products.
deterministic data-driven forecasting models suffer from issues with long-term stability and unphysical climate drift.
We propose a convolutional variational autoencoder-based data-driven model that is pre-trained on an imperfect climate model simulation.
arXiv Detail & Related papers (2022-05-09T23:52:37Z) - Back2Future: Leveraging Backfill Dynamics for Improving Real-time
Predictions in Future [73.03458424369657]
In real-time forecasting in public health, data collection is a non-trivial and demanding task.
'Backfill' phenomenon and its effect on model performance has been barely studied in the prior literature.
We formulate a novel problem and neural framework Back2Future that aims to refine a given model's predictions in real-time.
arXiv Detail & Related papers (2021-06-08T14:48:20Z) - Deep Switching Auto-Regressive Factorization:Application to Time Series
Forecasting [16.934920617960085]
DSARF approximates high dimensional data by a product variables between time dependent weights and spatially dependent factors.
DSARF is different from the state-of-the-art techniques in that it parameterizes the weights in terms of a deep switching vector auto-regressive factorization.
Our experiments attest the superior performance of DSARF in terms of long- and short-term prediction error, when compared with the state-of-the-art methods.
arXiv Detail & Related papers (2020-09-10T20:15:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.