CaFA: Global Weather Forecasting with Factorized Attention on Sphere

• URL: http://arxiv.org/abs/2405.07395v1
• Date: Sun, 12 May 2024 23:18:14 GMT
• Title: CaFA: Global Weather Forecasting with Factorized Attention on Sphere
• Authors: Zijie Li, Anthony Zhou, Saurabh Patil, Amir Barati Farimani,
• Abstract summary: We propose a factorized-attention-based model tailored for spherical geometries to mitigate this issue. The deterministic forecasting accuracy of the proposed model on $1.5circ$ and 0-7 days' lead time is on par with state-of-the-art purely data-driven machine learning weather prediction models.
• Score: 7.687215328455751
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate weather forecasting is crucial in various sectors, impacting decision-making processes and societal events. Data-driven approaches based on machine learning models have recently emerged as a promising alternative to numerical weather prediction models given their potential to capture physics of different scales from historical data and the significantly lower computational cost during the prediction stage. Renowned for its state-of-the-art performance across diverse domains, the Transformer model has also gained popularity in machine learning weather prediction. Yet applying Transformer architectures to weather forecasting, particularly on a global scale is computationally challenging due to the quadratic complexity of attention and the quadratic increase in spatial points as resolution increases. In this work, we propose a factorized-attention-based model tailored for spherical geometries to mitigate this issue. More specifically, it utilizes multi-dimensional factorized kernels that convolve over different axes where the computational complexity of the kernel is only quadratic to the axial resolution instead of overall resolution. The deterministic forecasting accuracy of the proposed model on $1.5^\circ$ and 0-7 days' lead time is on par with state-of-the-art purely data-driven machine learning weather prediction models. We also showcase the proposed model holds great potential to push forward the Pareto front of accuracy-efficiency for Transformer weather models, where it can achieve better accuracy with less computational cost compared to Transformer based models with standard attention.

Related papers

• Prithvi WxC: Foundation Model for Weather and Climate [2.9230020115516253]
  Prithvi WxC is a 2.3 billion parameter foundation model developed using 160 variables from the Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2). The model has been designed to accommodate large token counts to model weather phenomena in different topologies at fine resolutions. We test the model on a set of challenging downstream tasks namely: Autoregressive rollout forecasting, Downscaling, Gravity wave flux parameterization, and Extreme events estimation.
  arXiv  Detail & Related papers  (2024-09-20T15:53:17Z)
• Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling [55.13352174687475]
  This paper proposes a physics-AI hybrid model (i.e., WeatherGFT) which Generalizes weather forecasts to Finer-grained Temporal scales. Specifically, we employ a carefully designed PDE kernel to simulate physical evolution on a small time scale. We introduce a lead time-aware training framework to promote the generalization of the model at different lead times.
  arXiv  Detail & Related papers  (2024-05-22T16:21:02Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• FengWu-GHR: Learning the Kilometer-scale Medium-range Global Weather Forecasting [56.73502043159699]
  This work presents FengWu-GHR, the first data-driven global weather forecasting model running at the 0.09$circ$ horizontal resolution. It introduces a novel approach that opens the door for operating ML-based high-resolution forecasts by inheriting prior knowledge from a low-resolution model. The hindcast of weather prediction in 2022 indicates that FengWu-GHR is superior to the IFS-HRES.
  arXiv  Detail & Related papers  (2024-01-28T13:23:25Z)
• FengWu-4DVar: Coupling the Data-driven Weather Forecasting Model with 4D Variational Assimilation [67.20588721130623]
  We develop an AI-based cyclic weather forecasting system, FengWu-4DVar. FengWu-4DVar can incorporate observational data into the data-driven weather forecasting model. Experiments on the simulated observational dataset demonstrate that FengWu-4DVar is capable of generating reasonable analysis fields.
  arXiv  Detail & Related papers  (2023-12-16T02:07:56Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• Residual Corrective Diffusion Modeling for Km-scale Atmospheric Downscaling [58.456404022536425]
  State of the art for physical hazard prediction from weather and climate requires expensive km-scale numerical simulations driven by coarser resolution global inputs. Here, a generative diffusion architecture is explored for downscaling such global inputs to km-scale, as a cost-effective machine learning alternative. The model is trained to predict 2km data from a regional weather model over Taiwan, conditioned on a 25km global reanalysis.
  arXiv  Detail & Related papers  (2023-09-24T19:57:22Z)
• A case study of spatiotemporal forecasting techniques for weather forecasting [4.347494885647007]
  The correlations of real-world processes aretemporal, and the data generated by them exhibits both spatial and temporal evolution. Time series-based models are a viable alternative to numerical forecasts. We show that decompositiontemporal prediction models reduced computational costs while improving accuracy.
  arXiv  Detail & Related papers  (2022-09-29T13:47:02Z)
• Increasing the accuracy and resolution of precipitation forecasts using deep generative models [3.8073142980733]
  We train a conditional Generative Adversarial Network -- coined CorrectorGAN -- to produce ensembles of high-resolution, bias-corrected forecasts. CorrectorGAN, once trained, produces predictions in seconds on a single machine. Results raise exciting questions about the necessity of regional models, and whether data-driven downscaling and correction methods can be transferred to data-poor regions.
  arXiv  Detail & Related papers  (2022-03-23T09:45:12Z)
• A framework for probabilistic weather forecast post-processing across models and lead times using machine learning [3.1542695050861544]
  We show how to bridge the gap between sets of separate forecasts from NWP models and the 'ideal' forecast for decision support. We use Quantile Regression Forests to learn the error profile of each numerical model, and use these to apply empirically-derived probability distributions to forecasts. Second, we combine these probabilistic forecasts using quantile averaging. Third, we interpolate between the aggregate quantiles in order to generate a full predictive distribution.
  arXiv  Detail & Related papers  (2020-05-06T16:46:02Z)
• Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere [7.918783985810551]
  We present a significantly-improved data-driven global weather forecasting framework using a deep convolutional neural network (CNN) New developments in this framework include an offline volume-conservative mapping to a cubed-sphere grid. Our model is able to learn to forecast complex surface temperature patterns from few input atmospheric state variables.
  arXiv  Detail & Related papers  (2020-03-15T19:57:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.