LightWeather: Harnessing Absolute Positional Encoding to Efficient and Scalable Global Weather Forecasting

• URL: http://arxiv.org/abs/2408.09695v1
• Date: Mon, 19 Aug 2024 04:23:40 GMT
• Title: LightWeather: Harnessing Absolute Positional Encoding to Efficient and Scalable Global Weather Forecasting
• Authors: Yisong Fu, Fei Wang, Zezhi Shao, Chengqing Yu, Yujie Li, Zhao Chen, Zhulin An, Yongjun Xu,
• Abstract summary: We show that absolute positional encoding is what really works in Transformer-based weather forecasting models. We propose LightWeather, a lightweight and effective model for station-of-based global weather forecasting.
• Score: 21.048535830456363
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, Transformers have gained traction in weather forecasting for their capability to capture long-term spatial-temporal correlations. However, their complex architectures result in large parameter counts and extended training times, limiting their practical application and scalability to global-scale forecasting. This paper aims to explore the key factor for accurate weather forecasting and design more efficient solutions. Interestingly, our empirical findings reveal that absolute positional encoding is what really works in Transformer-based weather forecasting models, which can explicitly model the spatial-temporal correlations even without attention mechanisms. We theoretically prove that its effectiveness stems from the integration of geographical coordinates and real-world time features, which are intrinsically related to the dynamics of weather. Based on this, we propose LightWeather, a lightweight and effective model for station-based global weather forecasting. We employ absolute positional encoding and a simple MLP in place of other components of Transformer. With under 30k parameters and less than one hour of training time, LightWeather achieves state-of-the-art performance on global weather datasets compared to other advanced DL methods. The results underscore the superiority of integrating spatial-temporal knowledge over complex architectures, providing novel insights for DL in weather forecasting.

Related papers

• Advancing Meteorological Forecasting: AI-based Approach to Synoptic Weather Map Analysis [3.686808512438363]
  Our study proposes a novel preprocessing method and convolutional autoencoder model to improve the interpretation of synoptic weather maps. This model could recognize historical synoptic weather maps that nearly match current atmospheric conditions.
  arXiv  Detail & Related papers  (2024-11-08T07:46:50Z)
• WeatherFormer: Empowering Global Numerical Weather Forecasting with Space-Time Transformer [18.1906457042669]
  Numerical Weather Prediction (NWP) system is an infrastructure that exerts considerable impacts on modern society. Traditional NWP resolves complex partial differential equations with a huge computing cluster, resulting in tons of carbon emission. This work proposes a new transformer-based NWP framework, termed as WeatherFormer, to model complex-temporal atmosphere dynamics.
  arXiv  Detail & Related papers  (2024-09-21T07:02:31Z)
• How far are today's time-series models from real-world weather forecasting applications? [22.68937280154092]
  WEATHER-5K is a comprehensive collection of observational weather data that better reflects real-world scenarios. It enables a better training of models and a more accurate assessment of the real-world forecasting capabilities of TSF models. We provide researchers with a clear assessment of the gap between academic TSF models and real-world weather forecasting applications.
  arXiv  Detail & Related papers  (2024-06-20T15:18:52Z)
• WeatherFormer: A Pretrained Encoder Model for Learning Robust Weather Representations from Small Datasets [0.5735035463793009]
  WeatherFormer is a transformer encoder-based model designed to learn robust weather features from minimal observations. WeatherFormer was pretrained on a large pretraining dataset comprised of 39 years of satellite measurements across the Americas.
  arXiv  Detail & Related papers  (2024-05-22T17:43:46Z)
• Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling [55.13352174687475]
  This paper proposes a physics-AI hybrid model (i.e., WeatherGFT) which Generalizes weather forecasts to Finer-grained Temporal scales. Specifically, we employ a carefully designed PDE kernel to simulate physical evolution on a small time scale. We introduce a lead time-aware training framework to promote the generalization of the model at different lead times.
  arXiv  Detail & Related papers  (2024-05-22T16:21:02Z)
• Towards an end-to-end artificial intelligence driven global weather forecasting system [57.5191940978886]
  We present an AI-based data assimilation model, i.e., Adas, for global weather variables. We demonstrate that Adas can assimilate global observations to produce high-quality analysis, enabling the system operate stably for long term. We are the first to apply the methods to real-world scenarios, which is more challenging and has considerable practical application potential.
  arXiv  Detail & Related papers  (2023-12-18T09:05:28Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• ClimaX: A foundation model for weather and climate [51.208269971019504]
  ClimaX is a deep learning model for weather and climate science. It can be pre-trained with a self-supervised learning objective on climate datasets. It can be fine-tuned to address a breadth of climate and weather tasks.
  arXiv  Detail & Related papers  (2023-01-24T23:19:01Z)
• GraphCast: Learning skillful medium-range global weather forecasting [107.40054095223779]
  We introduce a machine learning-based method called "GraphCast", which can be trained directly from reanalysis data. It predicts hundreds of weather variables, over 10 days at 0.25 degree resolution globally, in under one minute. We show that GraphCast significantly outperforms the most accurate operational deterministic systems on 90% of 1380 verification targets.
  arXiv  Detail & Related papers  (2022-12-24T18:15:39Z)
• Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast [91.9372563527801]
  We present Pangu-Weather, a deep learning based system for fast and accurate global weather forecast. For the first time, an AI-based method outperforms state-of-the-art numerical weather prediction (NWP) methods in terms of accuracy. Pangu-Weather supports a wide range of downstream forecast scenarios, including extreme weather forecast and large-member ensemble forecast in real-time.
  arXiv  Detail & Related papers  (2022-11-03T17:19:43Z)
• Numerical Weather Forecasting using Convolutional-LSTM with Attention and Context Matcher Mechanisms [10.759556555869798]
  We introduce a novel deep learning architecture for forecasting high-resolution weather data. Our Weather Model achieves significant performance improvements compared to baseline deep learning models.
  arXiv  Detail & Related papers  (2021-02-01T08:30:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.