Ensemble data assimilation to diagnose AI-based weather prediction model: A case with ClimaX version 0.3.1

• URL: http://arxiv.org/abs/2407.17781v3
• Date: Thu, 3 Oct 2024 08:30:18 GMT
• Title: Ensemble data assimilation to diagnose AI-based weather prediction model: A case with ClimaX version 0.3.1
• Authors: Shunji Kotsuki, Kenta Shiraishi, Atsushi Okazaki,
• Abstract summary: This study proposes using ensemble data assimilation for diagnosing AI-based weather prediction models. Experiments with an AI-based model ClimaX demonstrated that the ensemble data assimilation cycled stably for the AI-based weather prediction model.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial intelligence (AI)-based weather prediction research is growing rapidly and has shown to be competitive with the advanced dynamic numerical weather prediction models. However, research combining AI-based weather prediction models with data assimilation remains limited partially because long-term sequential data assimilation cycles are required to evaluate data assimilation systems. This study proposes using ensemble data assimilation for diagnosing AI-based weather prediction models, and marked the first successful implementation of ensemble Kalman filter with AI-based weather prediction models. Our experiments with an AI-based model ClimaX demonstrated that the ensemble data assimilation cycled stably for the AI-based weather prediction model using covariance inflation and localization techniques within the ensemble Kalman filter. While ClimaX showed some limitations in capturing flow-dependent error covariance compared to dynamical models, the AI-based ensemble forecasts provided reasonable and beneficial error covariance in sparsely observed regions. In addition, ensemble data assimilation revealed that error growth based on ensemble ClimaX predictions was weaker than that of dynamical NWP models, leading to higher inflation factors. A series of experiments demonstrated that ensemble data assimilation can be used to diagnose properties of AI weather prediction models such as physical consistency and accurate error growth representation.

Related papers

• Weather Prediction Using CNN-LSTM for Time Series Analysis: A Case Study on Delhi Temperature Data [0.0]
  This study explores a hybrid CNN-LSTM model to enhance temperature forecasting accuracy for the Delhi region. We employed both direct and indirect methods, including comprehensive data preprocessing and exploratory analysis, to construct and train our model. Experimental results indicate that the CNN-LSTM model significantly outperforms traditional forecasting methods in terms of both accuracy and stability.
  arXiv  Detail & Related papers  (2024-09-14T11:06:07Z)
• A Benchmark for AI-based Weather Data Assimilation [10.100157158477145]
  We propose DABench, a benchmark constructed by simulated observations, real-world observations, and ERA5 reanalysis. Our experimental results demonstrate that the end-to-end weather forecasting system, integrating 4DVarFormerV2 and Sformer, can assimilate real-world observations. The proposed DABench will significantly advance research in AI-based DA, AI-based weather forecasting, and related domains.
  arXiv  Detail & Related papers  (2024-08-21T08:50:19Z)
• Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling [55.13352174687475]
  This paper proposes a physics-AI hybrid model (i.e., WeatherGFT) which Generalizes weather forecasts to Finer-grained Temporal scales. Specifically, we employ a carefully designed PDE kernel to simulate physical evolution on a small time scale. We introduce a lead time-aware training framework to promote the generalization of the model at different lead times.
  arXiv  Detail & Related papers  (2024-05-22T16:21:02Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• A Practical Probabilistic Benchmark for AI Weather Models [0.7978324349017066]
  We show that two leading AI weather models, i.e. GraphCast and Pangu, are tied on the probabilistic CRPS metric. We also reveal how multiple time-step loss functions, which many data-driven weather models have employed, are counter-productive.
  arXiv  Detail & Related papers  (2024-01-27T05:53:16Z)
• Towards an end-to-end artificial intelligence driven global weather forecasting system [57.5191940978886]
  We present an AI-based data assimilation model, i.e., Adas, for global weather variables. We demonstrate that Adas can assimilate global observations to produce high-quality analysis, enabling the system operate stably for long term. We are the first to apply the methods to real-world scenarios, which is more challenging and has considerable practical application potential.
  arXiv  Detail & Related papers  (2023-12-18T09:05:28Z)
• FengWu-4DVar: Coupling the Data-driven Weather Forecasting Model with 4D Variational Assimilation [67.20588721130623]
  We develop an AI-based cyclic weather forecasting system, FengWu-4DVar. FengWu-4DVar can incorporate observational data into the data-driven weather forecasting model. Experiments on the simulated observational dataset demonstrate that FengWu-4DVar is capable of generating reasonable analysis fields.
  arXiv  Detail & Related papers  (2023-12-16T02:07:56Z)
• Towards Learned Emulation of Interannual Water Isotopologue Variations in General Circulation Models [2.161227459325287]
  We investigate the possibility to replace the explicit physics-based simulation of oxygen isotopic composition in precipitation using machine learning methods. We implement convolutional neural networks (CNNs) based on the successful UNet architecture and test whether a spherical network architecture outperforms the naive approach of treating Earth's latitude-longitude grid as a flat image.
  arXiv  Detail & Related papers  (2023-01-31T07:54:52Z)
• ClimaX: A foundation model for weather and climate [51.208269971019504]
  ClimaX is a deep learning model for weather and climate science. It can be pre-trained with a self-supervised learning objective on climate datasets. It can be fine-tuned to address a breadth of climate and weather tasks.
  arXiv  Detail & Related papers  (2023-01-24T23:19:01Z)
• Spatiotemporal modeling of European paleoclimate using doubly sparse Gaussian processes [61.31361524229248]
  We build on recent scale sparsetemporal GPs to reduce the computational burden. We successfully employ such a doubly sparse GP to construct a probabilistic model of paleoclimate.
  arXiv  Detail & Related papers  (2022-11-15T14:15:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.