Finetuning AI Foundation Models to Develop Subgrid-Scale Parameterizations: A Case Study on Atmospheric Gravity Waves

• URL: http://arxiv.org/abs/2509.03816v1
• Date: Thu, 04 Sep 2025 02:05:54 GMT
• Title: Finetuning AI Foundation Models to Develop Subgrid-Scale Parameterizations: A Case Study on Atmospheric Gravity Waves
• Authors: Aman Gupta, Aditi Sheshadri, Sujit Roy, Johannes Schmude, Vishal Gaur, Wei Ji Leong, Manil Maskey, Rahul Ramachandran,
• Abstract summary: We present a new approach to developing machine learning parameterizations of small-scale climate processes by fine-tuning a pre-trained AI foundation model (FM)<n>A pre-trained encoder-decoder from a 2.3 billion parameter FM is fine-tuned to create a deep learning parameterization for atmospheric gravity waves (GWs)<n>A comparison of monthly averages and instantaneous evolution with a machine learning model baseline reveals superior predictive performance of the FM parameterization throughout the atmosphere.
• Score: 1.936101328226204
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Global climate models parameterize a range of atmospheric-oceanic processes like gravity waves, clouds, moist convection, and turbulence that cannot be sufficiently resolved. These subgrid-scale closures for unresolved processes are a leading source of model uncertainty. Here, we present a new approach to developing machine learning parameterizations of small-scale climate processes by fine-tuning a pre-trained AI foundation model (FM). FMs are largely unexplored in climate research. A pre-trained encoder-decoder from a 2.3 billion parameter FM (NASA and IBM Research's Prithvi WxC) -- which contains a latent probabilistic representation of atmospheric evolution -- is fine-tuned (or reused) to create a deep learning parameterization for atmospheric gravity waves (GWs). The parameterization captures GW effects for a coarse-resolution climate model by learning the fluxes from an atmospheric reanalysis with 10 times finer resolution. A comparison of monthly averages and instantaneous evolution with a machine learning model baseline (an Attention U-Net) reveals superior predictive performance of the FM parameterization throughout the atmosphere, even in regions excluded from pre-training. This performance boost is quantified using the Hellinger distance, which is 0.11 for the baseline and 0.06 for the fine-tuned model. Our findings emphasize the versatility and reusability of FMs, which could be used to accomplish a range of atmosphere- and climate-related applications, leading the way for the creation of observations-driven and physically accurate parameterizations for more earth-system processes.

Related papers

• Leveraging an Atmospheric Foundational Model for Subregional Sea Surface Temperature Forecasting [0.0]
  We adapt a deep learning model to predict sea temperature (SST) in the Canary Upwelling System.<n>By fine-tuning this model with high-resolution oceanographic reanalysis data, we demonstrate its ability to capture complex patterns.<n>The model successfully reproduces large-scale SST structures but faces challenges in capturing finer details in coastal regions.
  arXiv  Detail & Related papers  (2025-10-29T14:30:12Z)
• Diffusion models for probabilistic precipitation generation from atmospheric variables [1.6099193327384094]
  In Earth system models (ESMs), precipitation is not resolved explicitly, but represented by parameterizations.<n>We present a novel approach, based on generative machine learning, which integrates a conditional diffusion model with a UNet architecture.<n>Unlike traditional parameterizations, our framework efficiently produces ensemble predictions, capturing uncertainties in precipitation, and does not require fine-tuning by hand.
  arXiv  Detail & Related papers  (2025-04-01T00:21:31Z)
• Paraformer: Parameterization of Sub-grid Scale Processes Using Transformers [6.622831012413507]
  We propose a "memory-aware" Transformer-based model on ClimSim, the largest dataset ever created for climate parameterization.<n>Our results demonstrate that the proposed model successfully captures the complex non-linear dependencies in the sub-grid scale variables.
  arXiv  Detail & Related papers  (2024-12-21T20:21:52Z)
• MambaDS: Near-Surface Meteorological Field Downscaling with Topography Constrained Selective State Space Modeling [68.69647625472464]
  Downscaling, a crucial task in meteorological forecasting, enables the reconstruction of high-resolution meteorological states for target regions. Previous downscaling methods lacked tailored designs for meteorology and encountered structural limitations. We propose a novel model called MambaDS, which enhances the utilization of multivariable correlations and topography information.
  arXiv  Detail & Related papers  (2024-08-20T13:45:49Z)
• SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models [85.67096251281191]
  We present an innovative approach to model fusion called zero-shot Sparse MIxture of Low-rank Experts (SMILE) construction. SMILE allows for the upscaling of source models into an MoE model without extra data or further training. We conduct extensive experiments across diverse scenarios, such as image classification and text generation tasks, using full fine-tuning and LoRA fine-tuning.
  arXiv  Detail & Related papers  (2024-08-19T17:32:15Z)
• Machine Learning Global Simulation of Nonlocal Gravity Wave Propagation [1.3108798582758452]
  We present the first-ever global simulation of atmospheric mesoscale processes using machine learning (ML) models trained on the WINDSET dataset. Using an Attention U-Net-based architecture trained on globally resolved GW momentum, we illustrate the importance and effectiveness of global nonlocality.
  arXiv  Detail & Related papers  (2024-06-20T22:57:38Z)
• FengWu-GHR: Learning the Kilometer-scale Medium-range Global Weather Forecasting [56.73502043159699]
  This work presents FengWu-GHR, the first data-driven global weather forecasting model running at the 0.09$circ$ horizontal resolution. It introduces a novel approach that opens the door for operating ML-based high-resolution forecasts by inheriting prior knowledge from a low-resolution model. The hindcast of weather prediction in 2022 indicates that FengWu-GHR is superior to the IFS-HRES.
  arXiv  Detail & Related papers  (2024-01-28T13:23:25Z)
• Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method [66.80344502790231]
  We extend meteorological downscaling to arbitrary scattered station scales and establish a new benchmark and dataset. Inspired by data assimilation techniques, we integrate observational data into the downscaling process, providing multi-scale observational priors. Our proposed method outperforms other specially designed baseline models on multiple surface variables.
  arXiv  Detail & Related papers  (2024-01-22T14:02:56Z)
• Residual Corrective Diffusion Modeling for Km-scale Atmospheric Downscaling [58.456404022536425]
  State of the art for physical hazard prediction from weather and climate requires expensive km-scale numerical simulations driven by coarser resolution global inputs. Here, a generative diffusion architecture is explored for downscaling such global inputs to km-scale, as a cost-effective machine learning alternative. The model is trained to predict 2km data from a regional weather model over Taiwan, conditioned on a 25km global reanalysis.
  arXiv  Detail & Related papers  (2023-09-24T19:57:22Z)
• ClimaX: A foundation model for weather and climate [51.208269971019504]
  ClimaX is a deep learning model for weather and climate science. It can be pre-trained with a self-supervised learning objective on climate datasets. It can be fine-tuned to address a breadth of climate and weather tasks.
  arXiv  Detail & Related papers  (2023-01-24T23:19:01Z)
• Climate-Invariant Machine Learning [0.8831201550856289]
  Current climate models require representations of processes that occur at scales smaller than model grid size. Recent machine learning (ML) algorithms hold promise to improve such process representations, but tend to extrapolate poorly to climate regimes they were not trained on. We propose a new framework - termed "climate-invariant" ML - incorporating knowledge of climate processes into ML algorithms.
  arXiv  Detail & Related papers  (2021-12-14T07:02:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.