Generative Data Assimilation of Sparse Weather Station Observations at Kilometer Scales

• URL: http://arxiv.org/abs/2406.16947v1
• Date: Wed, 19 Jun 2024 10:28:11 GMT
• Title: Generative Data Assimilation of Sparse Weather Station Observations at Kilometer Scales
• Authors: Peter Manshausen, Yair Cohen, Jaideep Pathak, Mike Pritchard, Piyush Garg, Morteza Mardani, Karthik Kashinath, Simon Byrne, Noah Brenowitz,
• Abstract summary: We demonstrate the viability of score-based data assimilation in the context of realistically complex km-scale weather. By incorporating observations from 40 weather stations, 10% lower RMSEs on left-out stations are attained. It is a ripe time to explore extensions that combine increasingly ambitious regional state generators with an increasing set of in situ, ground-based, and satellite remote sensing data streams.
• Score: 5.453657018459705
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data assimilation of observational data into full atmospheric states is essential for weather forecast model initialization. Recently, methods for deep generative data assimilation have been proposed which allow for using new input data without retraining the model. They could also dramatically accelerate the costly data assimilation process used in operational regional weather models. Here, in a central US testbed, we demonstrate the viability of score-based data assimilation in the context of realistically complex km-scale weather. We train an unconditional diffusion model to generate snapshots of a state-of-the-art km-scale analysis product, the High Resolution Rapid Refresh. Then, using score-based data assimilation to incorporate sparse weather station data, the model produces maps of precipitation and surface winds. The generated fields display physically plausible structures, such as gust fronts, and sensitivity tests confirm learnt physics through multivariate relationships. Preliminary skill analysis shows the approach already outperforms a naive baseline of the High-Resolution Rapid Refresh system itself. By incorporating observations from 40 weather stations, 10\% lower RMSEs on left-out stations are attained. Despite some lingering imperfections such as insufficiently disperse ensemble DA estimates, we find the results overall an encouraging proof of concept, and the first at km-scale. It is a ripe time to explore extensions that combine increasingly ambitious regional state generators with an increasing set of in situ, ground-based, and satellite remote sensing data streams.

Related papers

• WeatherReal: A Benchmark Based on In-Situ Observations for Evaluating Weather Models [11.016845506758841]
  We introduce WeatherReal, a novel benchmark dataset for weather forecasting derived from global near-surface in-situ observations. This paper details the sources and processing methodologies underlying the dataset, and illustrates the advantage of in-situ observations in capturing hyper-local and extreme weather. Our work aims to advance the AI-based weather forecasting research towards a more application-focused and operation-ready approach.
  arXiv  Detail & Related papers  (2024-09-14T08:53:46Z)
• Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method [66.80344502790231]
  We extend meteorological downscaling to arbitrary scattered station scales and establish a new benchmark and dataset. Inspired by data assimilation techniques, we integrate observational data into the downscaling process, providing multi-scale observational priors. Our proposed method outperforms other specially designed baseline models on multiple surface variables.
  arXiv  Detail & Related papers  (2024-01-22T14:02:56Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• Residual Corrective Diffusion Modeling for Km-scale Atmospheric Downscaling [58.456404022536425]
  State of the art for physical hazard prediction from weather and climate requires expensive km-scale numerical simulations driven by coarser resolution global inputs. Here, a generative diffusion architecture is explored for downscaling such global inputs to km-scale, as a cost-effective machine learning alternative. The model is trained to predict 2km data from a regional weather model over Taiwan, conditioned on a 25km global reanalysis.
  arXiv  Detail & Related papers  (2023-09-24T19:57:22Z)
• Precipitation nowcasting with generative diffusion models [0.0]
  We study the efficacy of diffusion models in handling the task of precipitation nowcasting. Our work is conducted in comparison to the performance of well-established U-Net models.
  arXiv  Detail & Related papers  (2023-08-13T09:51:16Z)
• ClimaX: A foundation model for weather and climate [51.208269971019504]
  ClimaX is a deep learning model for weather and climate science. It can be pre-trained with a self-supervised learning objective on climate datasets. It can be fine-tuned to address a breadth of climate and weather tasks.
  arXiv  Detail & Related papers  (2023-01-24T23:19:01Z)
• GraphCast: Learning skillful medium-range global weather forecasting [107.40054095223779]
  We introduce a machine learning-based method called "GraphCast", which can be trained directly from reanalysis data. It predicts hundreds of weather variables, over 10 days at 0.25 degree resolution globally, in under one minute. We show that GraphCast significantly outperforms the most accurate operational deterministic systems on 90% of 1380 verification targets.
  arXiv  Detail & Related papers  (2022-12-24T18:15:39Z)
• A case study of spatiotemporal forecasting techniques for weather forecasting [4.347494885647007]
  The correlations of real-world processes aretemporal, and the data generated by them exhibits both spatial and temporal evolution. Time series-based models are a viable alternative to numerical forecasts. We show that decompositiontemporal prediction models reduced computational costs while improving accuracy.
  arXiv  Detail & Related papers  (2022-09-29T13:47:02Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)
• SmaAt-UNet: Precipitation Nowcasting using a Small Attention-UNet Architecture [5.28539620288341]
  We show that it is possible to produce an accurate precipitation nowcast using a data-driven neural network approach. We evaluate our approaches on a real-life datasets using precipitation maps from the region of the Netherlands and binary images of cloud coverage of France.
  arXiv  Detail & Related papers  (2020-07-08T20:33:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.