CirT: Global Subseasonal-to-Seasonal Forecasting with Geometry-inspired Transformer

• URL: http://arxiv.org/abs/2502.19750v1
• Date: Thu, 27 Feb 2025 04:26:23 GMT
• Title: CirT: Global Subseasonal-to-Seasonal Forecasting with Geometry-inspired Transformer
• Abstract summary: We propose the geometric-inspired Circular Transformer (CirT) to model the cyclic characteristic of the graticule.<n>Experiments on the Earth Reanalysis 5 (ERA5) reanalysis dataset demonstrate our model yields a significant improvement over the advanced data-driven models.
• Score: 47.65152457550307
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate Subseasonal-to-Seasonal (S2S) climate forecasting is pivotal for decision-making including agriculture planning and disaster preparedness but is known to be challenging due to its chaotic nature. Although recent data-driven models have shown promising results, their performance is limited by inadequate consideration of geometric inductive biases. Usually, they treat the spherical weather data as planar images, resulting in an inaccurate representation of locations and spatial relations. In this work, we propose the geometric-inspired Circular Transformer (CirT) to model the cyclic characteristic of the graticule, consisting of two key designs: (1) Decomposing the weather data by latitude into circular patches that serve as input tokens to the Transformer; (2) Leveraging Fourier transform in self-attention to capture the global information and model the spatial periodicity. Extensive experiments on the Earth Reanalysis 5 (ERA5) reanalysis dataset demonstrate our model yields a significant improvement over the advanced data-driven models, including PanguWeather and GraphCast, as well as skillful ECMWF systems. Additionally, we empirically show the effectiveness of our model designs and high-quality prediction over spatial and temporal dimensions.

Related papers

• Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting [52.53742727060265]
  Foundation models (FMs) for the Earth system learn statistical relationships between physical variables across massive datasets.<n>Here, we introduce Earth System Foundation Model (ESFM), a fully open model building on the 3Dwin UNet backbone of the pioneering model.<n>We present findings using dense gridded data (ERA5, CMIP6), regionally masked dense data, sparse gridded MODIS satellite data, and station data.<n>Results demonstrate competitive or superior performance relative to state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2026-04-20T11:40:39Z)
• VGGT-World: Transforming VGGT into an Autoregressive Geometry World Model [59.789011777899965]
  VGGT-World is a geometry world model that side-steps video generation entirely and instead forecasts the temporal evolution of frozen geometry-foundation-model features.<n>We show that VGGT-World significantly outperforms the strongest baselines in depth forecasting while running 3.6-5 times faster with only 0.43B trainable parameters.
  arXiv  Detail & Related papers  (2026-03-13T04:56:43Z)
• S$^2$Transformer: Scalable Structured Transformers for Global Station Weather Forecasting [67.93713728260646]
  Existing time series forecasting methods often ignore or unidirectionally model spatial correlation when conducting large-scale global station forecasting.<n>This contradicts the nature underlying observations of the global weather system limiting forecast performance.<n>We propose a novel Structured Spatial Attention in this paper.<n>It partitions the spatial graph into a set of subgraphs and instantiates Intra-subgraph Attention to learn local spatial correlation within each subgraph.<n>It aggregates nodes into subgraph representations for message passing among the subgraphs via Inter-subgraph Attention -- considering both spatial proximity and global correlation.
  arXiv  Detail & Related papers  (2025-09-10T05:33:28Z)
• Appa: Bending Weather Dynamics with Latent Diffusion Models for Global Data Assimilation [4.430758443755128]
  Appa is a score-based data assimilation model producing global atmospheric trajectories at 0.25-degree resolution and 1-hour intervals. Our results establish latent score-based data assimilation as a promising foundation for future global atmospheric modeling systems.
  arXiv  Detail & Related papers  (2025-04-25T22:14:29Z)
• TianQuan-S2S: A Subseasonal-to-Seasonal Global Weather Model via Incorporate Climatology State [23.8714341737652]
  Accurate Subseasonal-to-Seasonal (S2S) forecasting is vital for decision-making in agriculture, energy production, and emergency management.<n>Recent data-driven studies have shown promising results, but their performance is limited by the inadequate incorporation of climate states and a model tendency to degrade.<n>We propose TianQuan-S2S, a global S2S forecasting model that integrates initial weather states with climatological means via climatology into patch embedding and enhancing variability capture through an uncertainty-augmented Transformer.
  arXiv  Detail & Related papers  (2025-04-14T07:02:34Z)
• A Generative Framework for Probabilistic, Spatiotemporally Coherent Downscaling of Climate Simulation [23.504915709396204]
  We present a novel generative framework that uses a score-based diffusion model trained on high-resolution reanalysis data to capture the statistical properties of local weather dynamics.<n>We demonstrate that the model generates spatially and temporally coherent weather dynamics that align with global climate output.
  arXiv  Detail & Related papers  (2024-12-19T19:47:35Z)
• MambaDS: Near-Surface Meteorological Field Downscaling with Topography Constrained Selective State Space Modeling [68.69647625472464]
  Downscaling, a crucial task in meteorological forecasting, enables the reconstruction of high-resolution meteorological states for target regions. Previous downscaling methods lacked tailored designs for meteorology and encountered structural limitations. We propose a novel model called MambaDS, which enhances the utilization of multivariable correlations and topography information.
  arXiv  Detail & Related papers  (2024-08-20T13:45:49Z)
• A Scalable Real-Time Data Assimilation Framework for Predicting Turbulent Atmosphere Dynamics [8.012940782999975]
  We introduce a generic real-time data assimilation framework and demonstrate its end-to-end performance on the Frontier supercomputer. This framework comprises two primary modules: an ensemble score filter (EnSF) and a vision transformer-based surrogate. We demonstrate both the strong and weak scaling of our framework up to 1024 GPUs on the Exascale supercomputer, Frontier.
  arXiv  Detail & Related papers  (2024-07-16T20:44:09Z)
• CaFA: Global Weather Forecasting with Factorized Attention on Sphere [7.687215328455751]
  We propose a factorized-attention-based model tailored for spherical geometries to mitigate this issue. The deterministic forecasting accuracy of the proposed model on $1.5circ$ and 0-7 days' lead time is on par with state-of-the-art purely data-driven machine learning weather prediction models.
  arXiv  Detail & Related papers  (2024-05-12T23:18:14Z)
• HEAL-ViT: Vision Transformers on a spherical mesh for medium-range weather forecasting [0.14504054468850663]
  We present HEAL-ViT, a novel architecture that uses ViT models on a spherical mesh. HEAL-ViT produces weather forecasts that outperform the ECMWF IFS on key metrics.
  arXiv  Detail & Related papers  (2024-02-14T22:10:52Z)
• Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method [66.80344502790231]
  We extend meteorological downscaling to arbitrary scattered station scales and establish a new benchmark and dataset. Inspired by data assimilation techniques, we integrate observational data into the downscaling process, providing multi-scale observational priors. Our proposed method outperforms other specially designed baseline models on multiple surface variables.
  arXiv  Detail & Related papers  (2024-01-22T14:02:56Z)
• FengWu-4DVar: Coupling the Data-driven Weather Forecasting Model with 4D Variational Assimilation [67.20588721130623]
  We develop an AI-based cyclic weather forecasting system, FengWu-4DVar. FengWu-4DVar can incorporate observational data into the data-driven weather forecasting model. Experiments on the simulated observational dataset demonstrate that FengWu-4DVar is capable of generating reasonable analysis fields.
  arXiv  Detail & Related papers  (2023-12-16T02:07:56Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• ClimaX: A foundation model for weather and climate [51.208269971019504]
  ClimaX is a deep learning model for weather and climate science. It can be pre-trained with a self-supervised learning objective on climate datasets. It can be fine-tuned to address a breadth of climate and weather tasks.
  arXiv  Detail & Related papers  (2023-01-24T23:19:01Z)
• Earthformer: Exploring Space-Time Transformers for Earth System Forecasting [27.60569643222878]
  We propose Earthformer, a space-time Transformer for Earth system forecasting. The Transformer is based on a generic, flexible and efficient space-time attention block, named Cuboid Attention. Experiments on two real-world benchmarks about precipitation nowcasting and El Nino/Southerntemporaltion show Earthformer achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-07-12T20:52:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.