WeatherFormer: Empowering Global Numerical Weather Forecasting with Space-Time Transformer

• URL: http://arxiv.org/abs/2409.16321v1
• Date: Sat, 21 Sep 2024 07:02:31 GMT
• Title: WeatherFormer: Empowering Global Numerical Weather Forecasting with Space-Time Transformer
• Authors: Junchao Gong, Tao Han, Kang Chen, Lei Bai,
• Abstract summary: Numerical Weather Prediction (NWP) system is an infrastructure that exerts considerable impacts on modern society. Traditional NWP resolves complex partial differential equations with a huge computing cluster, resulting in tons of carbon emission. This work proposes a new transformer-based NWP framework, termed as WeatherFormer, to model complex-temporal atmosphere dynamics.
• Score: 18.1906457042669
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Numerical Weather Prediction (NWP) system is an infrastructure that exerts considerable impacts on modern society.Traditional NWP system, however, resolves it by solving complex partial differential equations with a huge computing cluster, resulting in tons of carbon emission. Exploring efficient and eco-friendly solutions for NWP attracts interest from Artificial Intelligence (AI) and earth science communities. To narrow the performance gap between the AI-based methods and physic predictor, this work proposes a new transformer-based NWP framework, termed as WeatherFormer, to model the complex spatio-temporal atmosphere dynamics and empowering the capability of data-driven NWP. WeatherFormer innovatively introduces the space-time factorized transformer blocks to decrease the parameters and memory consumption, in which Position-aware Adaptive Fourier Neural Operator (PAFNO) is proposed for location sensible token mixing. Besides, two data augmentation strategies are utilized to boost the performance and decrease training consumption. Extensive experiments on WeatherBench dataset show WeatherFormer achieves superior performance over existing deep learning methods and further approaches the most advanced physical model.

Related papers

• Efficient Localized Adaptation of Neural Weather Forecasting: A Case Study in the MENA Region [62.09891513612252]
  We focus on limited-area modeling and train our model specifically for localized region-level downstream tasks. We consider the MENA region due to its unique climatic challenges, where accurate localized weather forecasting is crucial for managing water resources, agriculture and mitigating the impacts of extreme weather events. Our study aims to validate the effectiveness of integrating parameter-efficient fine-tuning (PEFT) methodologies, specifically Low-Rank Adaptation (LoRA) and its variants, to enhance forecast accuracy, as well as training speed, computational resource utilization, and memory efficiency in weather and climate modeling for specific regions.
  arXiv  Detail & Related papers  (2024-09-11T19:31:56Z)
• LightWeather: Harnessing Absolute Positional Encoding to Efficient and Scalable Global Weather Forecasting [21.048535830456363]
  We show that absolute positional encoding is what really works in Transformer-based weather forecasting models. We propose LightWeather, a lightweight and effective model for station-of-based global weather forecasting.
  arXiv  Detail & Related papers  (2024-08-19T04:23:40Z)
• SFANet: Spatial-Frequency Attention Network for Weather Forecasting [54.470205739015434]
  Weather forecasting plays a critical role in various sectors, driving decision-making and risk management. Traditional methods often struggle to capture the complex dynamics of meteorological systems. We propose a novel framework designed to address these challenges and enhance the accuracy of weather prediction.
  arXiv  Detail & Related papers  (2024-05-29T08:00:15Z)
• Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling [55.13352174687475]
  This paper proposes a physics-AI hybrid model (i.e., WeatherGFT) which Generalizes weather forecasts to Finer-grained Temporal scales. Specifically, we employ a carefully designed PDE kernel to simulate physical evolution on a small time scale. We introduce a lead time-aware training framework to promote the generalization of the model at different lead times.
  arXiv  Detail & Related papers  (2024-05-22T16:21:02Z)
• ClimODE: Climate and Weather Forecasting with Physics-informed Neural ODEs [14.095897879222676]
  We present ClimODE, a continuous-time process that implements key principle of statistical mechanics. ClimODE models precise weather evolution with value-conserving dynamics, learning global weather transport as a neural flow. Our approach outperforms existing data-driven methods in global, regional forecasting with an order of magnitude smaller parameterization.
  arXiv  Detail & Related papers  (2024-04-15T06:38:21Z)
• Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere [53.63505583883769]
  We introduce Spherical FNOs (SFNOs) for learning operators on spherical geometries. SFNOs have important implications for machine learning-based simulation of climate dynamics.
  arXiv  Detail & Related papers  (2023-06-06T16:27:17Z)
• Long-term Wind Power Forecasting with Hierarchical Spatial-Temporal Transformer [112.12271800369741]
  Wind power is attracting increasing attention around the world due to its renewable, pollution-free, and other advantages. Accurate wind power forecasting (WPF) can effectively reduce power fluctuations in power system operations. Existing methods are mainly designed for short-term predictions and lack effective spatial-temporal feature augmentation.
  arXiv  Detail & Related papers  (2023-05-30T04:03:15Z)
• ClimaX: A foundation model for weather and climate [51.208269971019504]
  ClimaX is a deep learning model for weather and climate science. It can be pre-trained with a self-supervised learning objective on climate datasets. It can be fine-tuned to address a breadth of climate and weather tasks.
  arXiv  Detail & Related papers  (2023-01-24T23:19:01Z)
• Benchmark Dataset for Precipitation Forecasting by Post-Processing the Numerical Weather Prediction [11.52104902059751]
  We present a hybrid NWP-DL workflow to fill the gap between standalone NWP and DL approaches. Under this workflow, the NWP output is fed into a deep model, which post-processes the data to yield a refined precipitation forecast. We present a novel dataset focused on the Korean Peninsula, comprised of NWP predictions and AWS observations.
  arXiv  Detail & Related papers  (2022-06-30T12:41:32Z)
• Numerical Weather Forecasting using Convolutional-LSTM with Attention and Context Matcher Mechanisms [10.759556555869798]
  We introduce a novel deep learning architecture for forecasting high-resolution weather data. Our Weather Model achieves significant performance improvements compared to baseline deep learning models.
  arXiv  Detail & Related papers  (2021-02-01T08:30:42Z)
• Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere [7.918783985810551]
  We present a significantly-improved data-driven global weather forecasting framework using a deep convolutional neural network (CNN) New developments in this framework include an offline volume-conservative mapping to a cubed-sphere grid. Our model is able to learn to forecast complex surface temperature patterns from few input atmospheric state variables.
  arXiv  Detail & Related papers  (2020-03-15T19:57:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.