Timestep-Conditioned Transformers for Global Weather Forecasting

• URL: http://arxiv.org/abs/2608.06241v1
• Date: Thu, 06 Aug 2026 16:27:54 GMT
• Title: Timestep-Conditioned Transformers for Global Weather Forecasting
• Abstract summary: We present GEM-3, a probabilistic global weather model that addresses the trade-off through explicit multi-timestep inference.<n>With a single set of trained weights, the model timestep can be configured at inference time to balance predictability and usability across a broad forecast horizon.<n>We find that mixed-timestep training consistently improves rollout stability relative to timestep-specialist models.
• Score: 1.7632505349721652
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing machine-learning weather forecasting models rely on predetermined and fixed autoregressive timesteps. The choice of model timestep involves a fundamental trade-off: shorter timesteps (e.g. 1 to 6 hours) finely resolve atmospheric dynamics within the diurnal cycle but increase error accumulation for a given forecast horizon, while longer timesteps (e.g. 24 hours) reduce error accumulation but limit the usability of short-range forecasts where sub-daily predictability is high. In this work, we present GEM-3, a probabilistic global weather model that addresses this trade-off through explicit multi-timestep inference. With a single set of trained weights, the model timestep can be configured at inference time to balance predictability and usability across a broad forecast horizon. Additionally, we find that mixed-timestep training consistently improves rollout stability relative to timestep-specialist models. Under the hood, GEM-3 is a lightweight neighborhood-attention transformer with ~134M parameters on an equirectangular grid with a number of architectural advancements beyond its predecessor GEM-2. The result is a practical forecasting system that couples near-SOTA medium-range probabilistic skill, stable extended-range rollouts, efficient training and inference, and decision-relevant diagnostics.

Related papers

• Stochastically Perturbed Weights: Ensembles from Deterministic Machine-Learning Weather Models [0.0]
  Many machine-learning weather models (MLWMs) are deterministic, producing a single forecast with no estimate of its own uncertainty.<n>A growing family of trained-probabilistic models generate calibrated ensembles directly, at the price of a dedicated training run.<n>We ask how much uncertainty can be extracted from a deterministic checkpoint that already exists, without retraining it.
  arXiv  Detail & Related papers  (2026-09-08T08:18:40Z)
• A Hybrid LSTM--Vision Transformer Architecture for Predicting HRRR Forecast Errors [0.0]
  We develop a hybrid LSTM-Vision Transformer (LSTM-ViT) framework that combines temporal sequence learning from surface observations with atmospheric profiles from the New York State Mesonet profiler network.<n>Our research offers forecasters enhanced guidance regarding model bias and forecast confidence.
  arXiv  Detail & Related papers  (2026-06-17T12:51:46Z)
• LeapTS: Rethinking Time Series Forecasting as Adaptive Multi-Horizon Scheduling [74.94985663101906]
  We present LeapTS, a novel framework that reformulates time series forecasting as a dynamic scheduling process over the prediction horizon.<n>We show that LeapTS improves overall forecasting performance by at least 7.4% while achieving a 2.6$times$ to 5.3$times$ inference speedup over representative Transformer-based models.
  arXiv  Detail & Related papers  (2026-05-11T09:54:02Z)
• PnP-Corrector: A Universal Correction Framework for Coupled Spatiotemporal Forecasting [62.22445401483844]
  Existing methods are severely constrained by the bottleneck of compounding errors.<n>We propose a universal framework called -Cast-Corrector (Plug-and-Play Corrector)<n>Our method significantly enhances the long-term stability of coupled forecasting systems.
  arXiv  Detail & Related papers  (2026-05-09T13:12:33Z)
• Tyche: One Step Flow for Efficient Probabilistic Weather Forecasting [48.263481135566316]
  We present Tyche, a one-step conditional flow model for efficient probabilistic weather forecasting.<n>Experiments on ERA5 at 1.5$circ$ and 6-hour resolution show that our Tyche, using merely a single NFE, matches or exceeds the forecast skill and calibration of state-of-the-art multi-step generative baselines.
  arXiv  Detail & Related papers  (2026-05-07T20:25:28Z)
• A Physics-Guided AI Cascaded Corrector Model Significantly Extends Madden-Julian Oscillation Prediction Skill [12.768001894587576]
  We introduce a novel deep learning framework, the Physics-guided Cascaded Corrector for MJO (PCC-MJO)<n>PCC-MJO acts as a universal post-processor to correct MJO forecasts from dynamical models.<n>Our work provides a promising physically consistent, computationally efficient, and highly generalizable pathway to break through longstanding barriers in subseasonal forecasting.
  arXiv  Detail & Related papers  (2025-10-20T14:41:48Z)
• Skillful joint probabilistic weather forecasting from marginals [11.348323146521931]
  This paper presents FGN, a simple, scalable and flexible modeling approach which significantly outperforms the current state-of-the-art models.<n>It is trained directly to minimize the continuous rank probability score (CRPS) of per-location forecasts.<n>It produces state-of-the-art ensemble forecasts as measured by a range of deterministic and probabilistic metrics.
  arXiv  Detail & Related papers  (2025-06-12T14:50:47Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• GenCast: Diffusion-based ensemble forecasting for medium-range weather [10.845679586464026]
  We introduce GenCast, a probabilistic weather model with greater skill and speed than the top operational medium-range weather forecast in the world. GenCast generates an ensemble of 15-day global forecasts, at 12-hour steps and 0.25 degree latitude-longitude, for over 80 surface and atmospheric variables in 8 minutes. It has greater skill than ENS on 97.4% of 1320 targets we evaluated, and better predicts extreme weather, tropical cyclones, and wind power production.
  arXiv  Detail & Related papers  (2023-12-25T19:30:06Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• W-MAE: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting [7.610811907813171]
  We propose a Weather model with Masked AutoEncoder pre-training for weather forecasting. W-MAE is pre-trained in a self-supervised manner to reconstruct spatial correlations within meteorological variables. On the temporal scale, we fine-tune the pre-trained W-MAE to predict the future states of meteorological variables.
  arXiv  Detail & Related papers  (2023-04-18T06:25:11Z)
• Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting [98.57851612518758]
  Probabilistic time series forecasting involves estimating the distribution of future based on its history. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks. We show in experiments that our model produces accurate and sharp probabilistic forecasts.
  arXiv  Detail & Related papers  (2021-01-31T06:49:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.