S$^2$Transformer: Scalable Structured Transformers for Global Station Weather Forecasting

• URL: http://arxiv.org/abs/2509.19648v2
• Date: Thu, 25 Sep 2025 03:12:08 GMT
• Title: S$^2$Transformer: Scalable Structured Transformers for Global Station Weather Forecasting
• Authors: Hongyi Chen, Xiucheng Li, Xinyang Chen, Yun Cheng, Jing Li, Kehai Chen, Liqiang Nie,
• Abstract summary: Existing time series forecasting methods often ignore or unidirectionally model spatial correlation when conducting large-scale global station forecasting.<n>This contradicts the nature underlying observations of the global weather system limiting forecast performance.<n>We propose a novel Structured Spatial Attention in this paper.<n>It partitions the spatial graph into a set of subgraphs and instantiates Intra-subgraph Attention to learn local spatial correlation within each subgraph.<n>It aggregates nodes into subgraph representations for message passing among the subgraphs via Inter-subgraph Attention -- considering both spatial proximity and global correlation.
• Score: 67.93713728260646
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Global Station Weather Forecasting (GSWF) is a key meteorological research area, critical to energy, aviation, and agriculture. Existing time series forecasting methods often ignore or unidirectionally model spatial correlation when conducting large-scale global station forecasting. This contradicts the intrinsic nature underlying observations of the global weather system, limiting forecast performance. To address this, we propose a novel Spatial Structured Attention Block in this paper. It partitions the spatial graph into a set of subgraphs and instantiates Intra-subgraph Attention to learn local spatial correlation within each subgraph, and aggregates nodes into subgraph representations for message passing among the subgraphs via Inter-subgraph Attention -- considering both spatial proximity and global correlation. Building on this block, we develop a multiscale spatiotemporal forecasting model S$^2$Transformer by progressively expanding subgraph scales. The resulting model is both scalable and able to produce structured spatial correlation, and meanwhile, it is easy to implement. The experimental results show that it can achieve performance improvements up to 16.8% over time series forecasting baselines at low running costs.

Related papers

• STIPP: Space-time in situ postprocessing over the French Alps using proper scoring rules [1.887852744008007]
  Space-time in situ postprocessing (STIPP) is a machine learning model that generates consistent weather forecasts for a network of station locations.<n>By leveraging a proper scoring rule for training, STIPP contributes to ongoing work-driven atmospheric models supervised only with distribution marginals.
  arXiv  Detail & Related papers  (2026-01-06T10:07:20Z)
• PRISM: A hierarchical multiscale approach for time series forecasting [42.91635448262212]
  Real-world time series contain both global trends, local fine-grained structure, and features on multiple scales in between.<n>We present a new forecasting method, PRISM, that addresses this challenge through a learnable tree-based partitioning of the signal.<n> Experiments across benchmark datasets show that our method outperforms state-of-the-art methods for forecasting.
  arXiv  Detail & Related papers  (2025-12-31T14:51:12Z)
• Towards Scalable and Structured Spatiotemporal Forecasting [67.74910348499974]
  We propose a novel Spatial Balance Attention block fortemporal forecasting.<n>We partition the spatial graph into a set subgraphs and instantiate-graph Attention to learn local spatial correlation within each subgraph.<n>We develop a multiscale forecasting model by progressively increasing the subgraph scales.
  arXiv  Detail & Related papers  (2025-09-10T03:25:24Z)
• OneForecast: A Universal Framework for Global and Regional Weather Forecasting [67.61381313555091]
  We propose a global-regional nested weather forecasting framework (OneForecast) based on graph neural networks.<n>By combining a dynamic system perspective with multi-grid theory, we construct a multi-scale graph structure and densify the target region.<n>We introduce an adaptive messaging mechanism, using dynamic gating units, to deeply integrate node and edge features for more accurate extreme event forecasting.
  arXiv  Detail & Related papers  (2025-02-01T06:49:16Z)
• On the Integration of Spatial-Temporal Knowledge: A Lightweight Approach to Atmospheric Time Series Forecasting [24.119776558530983]
  The paper emphasizes the effectiveness of spatial-temporal knowledge integration over complex architectures, providing novel insights for atmospheric time series forecasting (ATSF)<n>With 10k parameters and one hour of training, STELLA achieves superior performance on five datasets compared to other advanced methods.
  arXiv  Detail & Related papers  (2024-08-19T04:23:40Z)
• Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method [66.80344502790231]
  We extend meteorological downscaling to arbitrary scattered station scales and establish a new benchmark and dataset. Inspired by data assimilation techniques, we integrate observational data into the downscaling process, providing multi-scale observational priors. Our proposed method outperforms other specially designed baseline models on multiple surface variables.
  arXiv  Detail & Related papers  (2024-01-22T14:02:56Z)
• SAMSGL: Series-Aligned Multi-Scale Graph Learning for Spatio-Temporal Forecasting [9.013416216828361]
  We present a Series-Aligned Multi-Scale Graph Learning (SGL) framework, aiming to enhance forecasting performance. In this work, we propose a series-aligned graph layer to facilitate the aggregation of non-delayed graph signals. We conduct experiments on meteorological and traffic forecasting datasets, which demonstrate its effectiveness and superiority.
  arXiv  Detail & Related papers  (2023-12-05T10:37:54Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• Federated Prompt Learning for Weather Foundation Models on Devices [37.88417074427373]
  On-device intelligence for weather forecasting uses local deep learning models to analyze weather patterns without centralized cloud computing. This paper propose Federated Prompt Learning for Weather Foundation Models on Devices (FedPoD) FedPoD enables devices to obtain highly customized models while maintaining communication efficiency.
  arXiv  Detail & Related papers  (2023-05-23T16:59:20Z)
• HiSTGNN: Hierarchical Spatio-temporal Graph Neural Networks for Weather Forecasting [13.317147032467306]
  We propose a novel Graph Hierarchical Spatio-Temporal Neural Network (HiSTGNN) to model cross-regional-temporal correlations among meteorological variables in multiple stations. Experimental results on three real-world meteorological datasets demonstrate the superior performance of HiSTGNN beyond 7 baselines. It reduces the errors by 4.2% to 11.6% especially compared to state-of-the-art weather forecasting method.
  arXiv  Detail & Related papers  (2022-01-22T17:30:46Z)
• Deep Autoregressive Models with Spectral Attention [74.08846528440024]
  We propose a forecasting architecture that combines deep autoregressive models with a Spectral Attention (SA) module. By characterizing in the spectral domain the embedding of the time series as occurrences of a random process, our method can identify global trends and seasonality patterns. Two spectral attention models, global and local to the time series, integrate this information within the forecast and perform spectral filtering to remove time series's noise.
  arXiv  Detail & Related papers  (2021-07-13T11:08:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.