On the Integration of Spatial-Temporal Knowledge: A Lightweight Approach to Atmospheric Time Series Forecasting

• URL: http://arxiv.org/abs/2408.09695v2
• Date: Wed, 24 Sep 2025 03:41:57 GMT
• Title: On the Integration of Spatial-Temporal Knowledge: A Lightweight Approach to Atmospheric Time Series Forecasting
• Authors: Yisong Fu, Fei Wang, Zezhi Shao, Boyu Diao, Lin Wu, Zhulin An, Chengqing Yu, Yujie Li, Yongjun Xu,
• Abstract summary: The paper emphasizes the effectiveness of spatial-temporal knowledge integration over complex architectures, providing novel insights for atmospheric time series forecasting (ATSF)<n>With 10k parameters and one hour of training, STELLA achieves superior performance on five datasets compared to other advanced methods.
• Score: 24.119776558530983
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transformers have gained attention in atmospheric time series forecasting (ATSF) for their ability to capture global spatial-temporal correlations. However, their complex architectures lead to excessive parameter counts and extended training times, limiting their scalability to large-scale forecasting. In this paper, we revisit ATSF from a theoretical perspective of atmospheric dynamics and uncover a key insight: spatial-temporal position embedding (STPE) can inherently model spatial-temporal correlations even without attention mechanisms. Its effectiveness arises from the integration of geographical coordinates and temporal features, which are intrinsically linked to atmospheric dynamics. Based on this, we propose STELLA, a Spatial-Temporal knowledge Embedded Lightweight modeL for ASTF, utilizing only STPE and an MLP architecture in place of Transformer layers. With 10k parameters and one hour of training, STELLA achieves superior performance on five datasets compared to other advanced methods. The paper emphasizes the effectiveness of spatial-temporal knowledge integration over complex architectures, providing novel insights for ATSF. The code is available at https://github.com/GestaltCogTeam/STELLA.

Related papers

• Scalable Spatio-Temporal SE(3) Diffusion for Long-Horizon Protein Dynamics [51.85385061275941]
  Molecular dynamics (MD) simulations remain the gold standard for studying protein dynamics.<n>Recent generative models have shown promise in accelerating simulations, yet they struggle with long-horizon generation.<n>We present STAR-MD, a scalable diffusion model that generates physically plausible protein trajectories over micro-scale timescales.
  arXiv  Detail & Related papers  (2026-02-02T14:13:28Z)
• A Deep Learning Approach for Spatio-Temporal Forecasting of InSAR Ground Deformation in Eastern Ireland [2.840858735842673]
  Monitoring ground displacement is crucial for urban infrastructure and mitigating geological hazards.<n>This paper introduces a novel deep learning framework that transforms sparse point measurements into a dense-temporal tensor.<n>Results demonstrate that the proposed architecture provides more accurate and spatially coherent forecasts.
  arXiv  Detail & Related papers  (2025-09-17T17:10:18Z)
• S$^2$Transformer: Scalable Structured Transformers for Global Station Weather Forecasting [67.93713728260646]
  Existing time series forecasting methods often ignore or unidirectionally model spatial correlation when conducting large-scale global station forecasting.<n>This contradicts the nature underlying observations of the global weather system limiting forecast performance.<n>We propose a novel Structured Spatial Attention in this paper.<n>It partitions the spatial graph into a set of subgraphs and instantiates Intra-subgraph Attention to learn local spatial correlation within each subgraph.<n>It aggregates nodes into subgraph representations for message passing among the subgraphs via Inter-subgraph Attention -- considering both spatial proximity and global correlation.
  arXiv  Detail & Related papers  (2025-09-10T05:33:28Z)
• ClimateLLM: Efficient Weather Forecasting via Frequency-Aware Large Language Models [13.740208247043258]
  We propose ClimateLLM, a foundation model for weather forecasting. It captures temporal dependencies via a cross-temporal and cross-spatial collaborative framework. It integrates frequency decomposition with Large Language Models to strengthen spatial and temporal modeling.
  arXiv  Detail & Related papers  (2025-02-16T09:57:50Z)
• HyperFLINT: Hypernetwork-based Flow Estimation and Temporal Interpolation for Scientific Ensemble Visualization [26.472939569860607]
  HyperFLINT is a novel deep learning-based approach for estimating flow fields, temporally interpolating fields, and facilitating parameter space exploration in ensemble data.<n>A series of experiments demonstrates HyperFLINT's significantly improved performance in flow field estimation, as well as its potential in enabling parameter space exploration.
  arXiv  Detail & Related papers  (2024-12-05T12:01:20Z)
• Cross Space and Time: A Spatio-Temporal Unitized Model for Traffic Flow Forecasting [16.782154479264126]
  Predicting backbone-temporal traffic flow presents challenges due to complex interactions between temporal factors. Existing approaches address these dimensions in isolation, neglecting their critical interdependencies. In this paper, we introduce Sanonymous-Temporal Unitized Unitized Cell (ASTUC), a unified framework designed to capture both spatial and temporal dependencies.
  arXiv  Detail & Related papers  (2024-11-14T07:34:31Z)
• Advancing Meteorological Forecasting: AI-based Approach to Synoptic Weather Map Analysis [3.686808512438363]
  Our study proposes a novel preprocessing method and convolutional autoencoder model to improve the interpretation of synoptic weather maps. This model could recognize historical synoptic weather maps that nearly match current atmospheric conditions.
  arXiv  Detail & Related papers  (2024-11-08T07:46:50Z)
• WeatherFormer: Empowering Global Numerical Weather Forecasting with Space-Time Transformer [18.1906457042669]
  Numerical Weather Prediction (NWP) system is an infrastructure that exerts considerable impacts on modern society. Traditional NWP resolves complex partial differential equations with a huge computing cluster, resulting in tons of carbon emission. This work proposes a new transformer-based NWP framework, termed as WeatherFormer, to model complex-temporal atmosphere dynamics.
  arXiv  Detail & Related papers  (2024-09-21T07:02:31Z)
• How far are today's time-series models from real-world weather forecasting applications? [22.68937280154092]
  WEATHER-5K is a comprehensive collection of observational weather data that better reflects real-world scenarios. It enables a better training of models and a more accurate assessment of the real-world forecasting capabilities of TSF models. We provide researchers with a clear assessment of the gap between academic TSF models and real-world weather forecasting applications.
  arXiv  Detail & Related papers  (2024-06-20T15:18:52Z)
• SFANet: Spatial-Frequency Attention Network for Weather Forecasting [54.470205739015434]
  Weather forecasting plays a critical role in various sectors, driving decision-making and risk management. Traditional methods often struggle to capture the complex dynamics of meteorological systems. We propose a novel framework designed to address these challenges and enhance the accuracy of weather prediction.
  arXiv  Detail & Related papers  (2024-05-29T08:00:15Z)
• WeatherFormer: A Pretrained Encoder Model for Learning Robust Weather Representations from Small Datasets [0.5735035463793009]
  WeatherFormer is a transformer encoder-based model designed to learn robust weather features from minimal observations. WeatherFormer was pretrained on a large pretraining dataset comprised of 39 years of satellite measurements across the Americas.
  arXiv  Detail & Related papers  (2024-05-22T17:43:46Z)
• Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling [55.13352174687475]
  This paper proposes a physics-AI hybrid model (i.e., WeatherGFT) which Generalizes weather forecasts to Finer-grained Temporal scales. Specifically, we employ a carefully designed PDE kernel to simulate physical evolution on a small time scale. We introduce a lead time-aware training framework to promote the generalization of the model at different lead times.
  arXiv  Detail & Related papers  (2024-05-22T16:21:02Z)
• Towards an end-to-end artificial intelligence driven global weather forecasting system [57.5191940978886]
  We present an AI-based data assimilation model, i.e., Adas, for global weather variables. We demonstrate that Adas can assimilate global observations to produce high-quality analysis, enabling the system operate stably for long term. We are the first to apply the methods to real-world scenarios, which is more challenging and has considerable practical application potential.
  arXiv  Detail & Related papers  (2023-12-18T09:05:28Z)
• Earthfarseer: Versatile Spatio-Temporal Dynamical Systems Modeling in One Model [23.875981403451256]
  EarthFarseer is a framework that combines parallel local convolutions and global Fourier-based transformer architectures. Our proposal demonstrates strong adaptability across various convergence and datasets, with fast and better local fidelity in long time-steps predictions.
  arXiv  Detail & Related papers  (2023-12-13T07:20:24Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• Disentangling Spatial and Temporal Learning for Efficient Image-to-Video Transfer Learning [59.26623999209235]
  We present DiST, which disentangles the learning of spatial and temporal aspects of videos. The disentangled learning in DiST is highly efficient because it avoids the back-propagation of massive pre-trained parameters. Extensive experiments on five benchmarks show that DiST delivers better performance than existing state-of-the-art methods by convincing gaps.
  arXiv  Detail & Related papers  (2023-09-14T17:58:33Z)
• Long-term Wind Power Forecasting with Hierarchical Spatial-Temporal Transformer [112.12271800369741]
  Wind power is attracting increasing attention around the world due to its renewable, pollution-free, and other advantages. Accurate wind power forecasting (WPF) can effectively reduce power fluctuations in power system operations. Existing methods are mainly designed for short-term predictions and lack effective spatial-temporal feature augmentation.
  arXiv  Detail & Related papers  (2023-05-30T04:03:15Z)
• ClimaX: A foundation model for weather and climate [51.208269971019504]
  ClimaX is a deep learning model for weather and climate science. It can be pre-trained with a self-supervised learning objective on climate datasets. It can be fine-tuned to address a breadth of climate and weather tasks.
  arXiv  Detail & Related papers  (2023-01-24T23:19:01Z)
• GraphCast: Learning skillful medium-range global weather forecasting [107.40054095223779]
  We introduce a machine learning-based method called "GraphCast", which can be trained directly from reanalysis data. It predicts hundreds of weather variables, over 10 days at 0.25 degree resolution globally, in under one minute. We show that GraphCast significantly outperforms the most accurate operational deterministic systems on 90% of 1380 verification targets.
  arXiv  Detail & Related papers  (2022-12-24T18:15:39Z)
• Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast [91.9372563527801]
  We present Pangu-Weather, a deep learning based system for fast and accurate global weather forecast. For the first time, an AI-based method outperforms state-of-the-art numerical weather prediction (NWP) methods in terms of accuracy. Pangu-Weather supports a wide range of downstream forecast scenarios, including extreme weather forecast and large-member ensemble forecast in real-time.
  arXiv  Detail & Related papers  (2022-11-03T17:19:43Z)
• Numerical Weather Forecasting using Convolutional-LSTM with Attention and Context Matcher Mechanisms [10.759556555869798]
  We introduce a novel deep learning architecture for forecasting high-resolution weather data. Our Weather Model achieves significant performance improvements compared to baseline deep learning models.
  arXiv  Detail & Related papers  (2021-02-01T08:30:42Z)
• Physics-informed Tensor-train ConvLSTM for Volumetric Velocity Forecasting of Loop Current [6.016102212809306]
  Loop Current is a weekly forecast of velocity, vertical structure, and duration of the Loop Current (LC) in the Gulf of Mexico. This paper shows its effectiveness beyond video prediction, to a novel Physics-informed spatial-train ConvLSTM for temporal sequences of 3D geospatial data forecasting.
  arXiv  Detail & Related papers  (2020-08-04T19:55:57Z)
• A Spatial-Temporal Attentive Network with Spatial Continuity for Trajectory Prediction [74.00750936752418]
  We propose a novel model named spatial-temporal attentive network with spatial continuity (STAN-SC) First, spatial-temporal attention mechanism is presented to explore the most useful and important information. Second, we conduct a joint feature sequence based on the sequence and instant state information to make the generative trajectories keep spatial continuity.
  arXiv  Detail & Related papers  (2020-03-13T04:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.