ClimaX: A foundation model for weather and climate
- URL: http://arxiv.org/abs/2301.10343v5
- Date: Mon, 18 Dec 2023 18:16:12 GMT
- Title: ClimaX: A foundation model for weather and climate
- Authors: Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K. Gupta,
Aditya Grover
- Abstract summary: ClimaX is a deep learning model for weather and climate science.
It can be pre-trained with a self-supervised learning objective on climate datasets.
It can be fine-tuned to address a breadth of climate and weather tasks.
- Score: 51.208269971019504
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Most state-of-the-art approaches for weather and climate modeling are based
on physics-informed numerical models of the atmosphere. These approaches aim to
model the non-linear dynamics and complex interactions between multiple
variables, which are challenging to approximate. Additionally, many such
numerical models are computationally intensive, especially when modeling the
atmospheric phenomenon at a fine-grained spatial and temporal resolution.
Recent data-driven approaches based on machine learning instead aim to directly
solve a downstream forecasting or projection task by learning a data-driven
functional mapping using deep neural networks. However, these networks are
trained using curated and homogeneous climate datasets for specific
spatiotemporal tasks, and thus lack the generality of numerical models. We
develop and demonstrate ClimaX, a flexible and generalizable deep learning
model for weather and climate science that can be trained using heterogeneous
datasets spanning different variables, spatio-temporal coverage, and physical
groundings. ClimaX extends the Transformer architecture with novel encoding and
aggregation blocks that allow effective use of available compute while
maintaining general utility. ClimaX is pre-trained with a self-supervised
learning objective on climate datasets derived from CMIP6. The pre-trained
ClimaX can then be fine-tuned to address a breadth of climate and weather
tasks, including those that involve atmospheric variables and spatio-temporal
scales unseen during pretraining. Compared to existing data-driven baselines,
we show that this generality in ClimaX results in superior performance on
benchmarks for weather forecasting and climate projections, even when
pretrained at lower resolutions and compute budgets. The source code is
available at https://github.com/microsoft/ClimaX.
Related papers
- A Scalable Real-Time Data Assimilation Framework for Predicting Turbulent Atmosphere Dynamics [8.012940782999975]
We introduce a generic real-time data assimilation framework and demonstrate its end-to-end performance on the Frontier supercomputer.
This framework comprises two primary modules: an ensemble score filter (EnSF) and a vision transformer-based surrogate.
We demonstrate both the strong and weak scaling of our framework up to 1024 GPUs on the Exascale supercomputer, Frontier.
arXiv Detail & Related papers (2024-07-16T20:44:09Z) - Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling [55.13352174687475]
This paper proposes a physics-AI hybrid model (i.e., WeatherGFT) which Generalizes weather forecasts to Finer-grained Temporal scales.
Specifically, we employ a carefully designed PDE kernel to simulate physical evolution on a small time scale.
We introduce a lead time-aware training framework to promote the generalization of the model at different lead times.
arXiv Detail & Related papers (2024-05-22T16:21:02Z) - ClimODE: Climate and Weather Forecasting with Physics-informed Neural ODEs [14.095897879222676]
We present ClimODE, a continuous-time process that implements key principle of statistical mechanics.
ClimODE models precise weather evolution with value-conserving dynamics, learning global weather transport as a neural flow.
Our approach outperforms existing data-driven methods in global, regional forecasting with an order of magnitude smaller parameterization.
arXiv Detail & Related papers (2024-04-15T06:38:21Z) - Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies.
Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
arXiv Detail & Related papers (2023-11-30T08:22:08Z) - Pushing the Limits of Pre-training for Time Series Forecasting in the
CloudOps Domain [54.67888148566323]
We introduce three large-scale time series forecasting datasets from the cloud operations domain.
We show it is a strong zero-shot baseline and benefits from further scaling, both in model and dataset size.
Accompanying these datasets and results is a suite of comprehensive benchmark results comparing classical and deep learning baselines to our pre-trained method.
arXiv Detail & Related papers (2023-10-08T08:09:51Z) - Exploring the Application of Large-scale Pre-trained Models on Adverse
Weather Removal [97.53040662243768]
We propose a CLIP embedding module to make the network handle different weather conditions adaptively.
This module integrates the sample specific weather prior extracted by CLIP image encoder together with the distribution specific information learned by a set of parameters.
arXiv Detail & Related papers (2023-06-15T10:06:13Z) - Climate Intervention Analysis using AI Model Guided by Statistical
Physics Principles [6.824166358727082]
We propose a novel solution by utilizing a principle from statistical physics known as the Fluctuation-Dissipation Theorem (FDT)
By leveraging, we are able to extract information encoded in a large dataset produced by Earth System Models.
Our model, AiBEDO, is capable of capturing the complex, multi-timescale effects of radiation perturbations on global and regional surface climate.
arXiv Detail & Related papers (2023-02-07T05:09:10Z) - Deep Learning Based Cloud Cover Parameterization for ICON [55.49957005291674]
We train NN based cloud cover parameterizations with coarse-grained data based on realistic regional and global ICON simulations.
Globally trained NNs can reproduce sub-grid scale cloud cover of the regional simulation.
We identify an overemphasis on specific humidity and cloud ice as the reason why our column-based NN cannot perfectly generalize from the global to the regional coarse-grained data.
arXiv Detail & Related papers (2021-12-21T16:10:45Z) - SubseasonalClimateUSA: A Dataset for Subseasonal Forecasting and
Benchmarking [20.442879707675115]
SubseasonalClimateUSA is a curated dataset for training and benchmarking subseasonal forecasting models in the United States.
We use this dataset to benchmark a diverse suite of models, including operational dynamical models, classical meteorological baselines, and ten state-of-the-art machine learning and deep learning-based methods from the literature.
arXiv Detail & Related papers (2021-09-21T18:42:10Z) - Numerical Weather Forecasting using Convolutional-LSTM with Attention
and Context Matcher Mechanisms [10.759556555869798]
We introduce a novel deep learning architecture for forecasting high-resolution weather data.
Our Weather Model achieves significant performance improvements compared to baseline deep learning models.
arXiv Detail & Related papers (2021-02-01T08:30:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.