On the Predictive Skill of Artificial Intelligence-based Weather Models for Extreme Events using Uncertainty Quantification

• URL: http://arxiv.org/abs/2511.17176v1
• Date: Fri, 21 Nov 2025 11:54:22 GMT
• Title: On the Predictive Skill of Artificial Intelligence-based Weather Models for Extreme Events using Uncertainty Quantification
• Authors: Rodrigo Almeida, Noelia Otero, Miguel-Ángel Fernández-Torres, Jackie Ma,
• Abstract summary: This study investigates how state of the art deterministic artificial intelligence based models respond to initial-condition perturbations.<n>We generate 50 member ensembles for two major events in August 2022: the Pakistan floods and the China heatwave.<n>Results show that flow dependent perturbations produce the most realistic ensemble spread and highest probabilistic skill, narrowing but not closing the performance gap with numerical weather prediction ensembles.
• Score: 0.34875901464487297
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate prediction of extreme weather events remains a major challenge for artificial intelligence based weather prediction systems. While deterministic models such as FuXi, GraphCast, and SFNO have achieved competitive forecast skill relative to numerical weather prediction, their ability to represent uncertainty and capture extremes is still limited. This study investigates how state of the art deterministic artificial intelligence based models respond to initial-condition perturbations and evaluates the resulting ensembles in forecasting extremes. Using three perturbation strategies (Gaussian noise, Hemispheric Centered Bred Vectors, and Huge Ensembles), we generate 50 member ensembles for two major events in August 2022: the Pakistan floods and the China heatwave. Ensemble skill is assessed against ERA5 and compared with IFS ENS and the probabilistic AIFSENS model using deterministic and probabilistic metrics. Results show that flow dependent perturbations produce the most realistic ensemble spread and highest probabilistic skill, narrowing but not closing the performance gap with numerical weather prediction ensembles. Across variables, artificial intelligence based weather models capture temperature extremes more effectively than precipitation. These findings demonstrate that input perturbations can extend deterministic models toward probabilistic forecasting, paving the way for approaches that combine flow dependent perturbations with generative or latent-space uncertainty modeling for reliable artificial intelligence-driven early warning systems.

Related papers

• Forecasting Fails: Unveiling Evasion Attacks in Weather Prediction Models [60.728124907335]
  This work introduces Weather Adaptive Adversarial Perturbation Optimization (WAAPO), a novel framework for generating targeted adversarial perturbations.<n>WAAPO achieves this by incorporating constraints for channel sparsity, spatial localization, and smoothness, ensuring that perturbations remain physically realistic and imperceptible.<n>Our experiments highlight critical vulnerabilities in AI-driven forecasting models, where small perturbations to initial conditions can result in significant deviations.
  arXiv  Detail & Related papers  (2025-12-09T17:20:56Z)
• Bridging the Gap Between Bayesian Deep Learning and Ensemble Weather Forecasts [100.26854618129039]
  Weather forecasting is fundamentally challenged by the chaotic nature of the atmosphere.<n>Recent advances in Bayesian Deep Learning (BDL) offer a promising but often disconnected alternative.<n>We bridge these paradigms through a unified hybrid BDL framework for ensemble weather forecasting.
  arXiv  Detail & Related papers  (2025-11-18T07:49:52Z)
• Lightning-Fast Convective Outlooks: Predicting Severe Convective Environments with Global AI-based Weather Models [0.08271752505511926]
  Severe convective storms are among the most dangerous weather phenomena and accurate forecasts mitigate their impacts. Recently released suite of AI-based weather models produces medium-range forecasts within seconds. We assess the forecast skill of three top-performing AI-models for convective parameters against reanalysis and ECMWF's operational numerical weather prediction model IFS.
  arXiv  Detail & Related papers  (2024-06-13T07:46:03Z)
• Uncertainty quantification for data-driven weather models [0.0]
  We study and compare uncertainty quantification methods to generate probabilistic weather forecasts from a state-of-the-art deterministic data-driven weather model, Pangu-Weather.<n>Specifically, we compare approaches for quantifying forecast uncertainty based on generating ensemble forecasts via perturbations to the initial conditions.<n>In a case study on medium-range forecasts of selected weather variables over Europe, the probabilistic forecasts obtained by using the Pangu-Weather model in concert with uncertainty quantification methods show promising results.
  arXiv  Detail & Related papers  (2024-03-20T10:07:51Z)
• Weather Prediction with Diffusion Guided by Realistic Forecast Processes [49.07556359513563]
  We introduce a novel method that applies diffusion models (DM) for weather forecasting. Our method can achieve both direct and iterative forecasting with the same modeling framework. The flexibility and controllability of our model empowers a more trustworthy DL system for the general weather community.
  arXiv  Detail & Related papers  (2024-02-06T21:28:42Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• A Practical Probabilistic Benchmark for AI Weather Models [0.7978324349017066]
  We show that two leading AI weather models, i.e. GraphCast and Pangu, are tied on the probabilistic CRPS metric. We also reveal how multiple time-step loss functions, which many data-driven weather models have employed, are counter-productive.
  arXiv  Detail & Related papers  (2024-01-27T05:53:16Z)
• SEEDS: Emulation of Weather Forecast Ensembles with Diffusion Models [13.331224394143117]
  Uncertainty quantification is crucial to decision-making. dominant approach to representing uncertainty in weather forecasting is to generate an ensemble of forecasts. We propose to amortize the computational cost by emulating these forecasts with deep generative diffusion models learned from historical data.
  arXiv  Detail & Related papers  (2023-06-24T22:00:06Z)
• Uncertainty estimation of pedestrian future trajectory using Bayesian approximation [137.00426219455116]
  Under dynamic traffic scenarios, planning based on deterministic predictions is not trustworthy. The authors propose to quantify uncertainty during forecasting using approximation which deterministic approaches fail to capture. The effect of dropout weights and long-term prediction on future state uncertainty has been studied.
  arXiv  Detail & Related papers  (2022-05-04T04:23:38Z)
• A generative adversarial network approach to (ensemble) weather prediction [91.3755431537592]
  We use a conditional deep convolutional generative adversarial network to predict the geopotential height of the 500 hPa pressure level, the two-meter temperature and the total precipitation for the next 24 hours over Europe. The proposed models are trained on 4 years of ERA5 reanalysis data from 2015-2018 with the goal to predict the associated meteorological fields in 2019.
  arXiv  Detail & Related papers  (2020-06-13T20:53:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.