Uncertainty Quantification Techniques for Space Weather Modeling: Thermospheric Density Application

• URL: http://arxiv.org/abs/2201.02067v1
• Date: Thu, 6 Jan 2022 14:17:50 GMT
• Title: Uncertainty Quantification Techniques for Space Weather Modeling: Thermospheric Density Application
• Authors: Richard J. Licata and Piyush M. Mehta
• Abstract summary: We propose two techniques to develop nonlinear ML models to predict thermospheric density. We show the performance for models trained on local and global datasets. We achieve errors of 11% on independent test data with well-calibrated uncertainty estimates.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning (ML) has often been applied to space weather (SW) problems in recent years. SW originates from solar perturbations and is comprised of the resulting complex variations they cause within the systems between the Sun and Earth. These systems are tightly coupled and not well understood. This creates a need for skillful models with knowledge about the confidence of their predictions. One example of such a dynamical system is the thermosphere, the neutral region of Earth's upper atmosphere. Our inability to forecast it has severe repercussions in the context of satellite drag and collision avoidance operations for objects in low Earth orbit. Even with (assumed) perfect driver forecasts, our incomplete knowledge of the system results in often inaccurate neutral mass density predictions. Continuing efforts are being made to improve model accuracy, but density models rarely provide estimates of uncertainty. In this work, we propose two techniques to develop nonlinear ML models to predict thermospheric density while providing calibrated uncertainty estimates: Monte Carlo (MC) dropout and direct prediction of the probability distribution, both using the negative logarithm of predictive density (NLPD) loss function. We show the performance for models trained on local and global datasets. This shows that NLPD provides similar results for both techniques but the direct probability method has a much lower computational cost. For the global model regressed on the SET HASDM density database, we achieve errors of 11% on independent test data with well-calibrated uncertainty estimates. Using an in-situ CHAMP density dataset, both techniques provide test error on the order of 13%. The CHAMP models (on independent data) are within 2% of perfect calibration for all prediction intervals tested. This model can also be used to obtain global predictions with uncertainties at a given epoch.

Related papers

• CoDiCast: Conditional Diffusion Model for Global Weather Prediction with Uncertainty Quantification [23.605859261510407]
  Traditional ensemble numerical weather prediction is computationally intensive. Most existing machine learning-based weather prediction approaches are efficient and accurate. We propose CoDiCast, a conditional diffusion model to generate accurate global weather prediction.
  arXiv  Detail & Related papers  (2024-09-09T18:18:47Z)
• Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events [0.1747623282473278]
  We compare machine learning weather prediction models and ECMWF's high-resolution forecast system. We find that ML weather prediction models locally achieve similar accuracy to HRES on the record-shattering Pacific Northwest heatwave. We also highlight structural differences in how the errors of HRES and the ML models build up to that event.
  arXiv  Detail & Related papers  (2024-04-26T18:18:25Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• Ensemble models outperform single model uncertainties and predictions for operator-learning of hypersonic flows [43.148818844265236]
  Training scientific machine learning (SciML) models on limited high-fidelity data offers one approach to rapidly predict behaviors for situations that have not been seen before. High-fidelity data is itself in limited quantity to validate all outputs of the SciML model in unexplored input space. We extend a DeepONet using three different uncertainty mechanisms: mean-variance estimation, evidential uncertainty, and ensembling.
  arXiv  Detail & Related papers  (2023-10-31T18:07:29Z)
• Residual Corrective Diffusion Modeling for Km-scale Atmospheric Downscaling [58.456404022536425]
  State of the art for physical hazard prediction from weather and climate requires expensive km-scale numerical simulations driven by coarser resolution global inputs. Here, a generative diffusion architecture is explored for downscaling such global inputs to km-scale, as a cost-effective machine learning alternative. The model is trained to predict 2km data from a regional weather model over Taiwan, conditioned on a 25km global reanalysis.
  arXiv  Detail & Related papers  (2023-09-24T19:57:22Z)
• Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy Estimation [51.809741427975105]
  This work aims to assess how well a model performs under distribution shifts without using labels. We use the nuclear norm that has been shown to be effective in characterizing both properties. We show that the nuclear norm is more accurate and robust in accuracy than existing methods.
  arXiv  Detail & Related papers  (2023-02-02T13:30:48Z)
• Science through Machine Learning: Quantification of Poststorm Thermospheric Cooling [0.0]
  We develop machine learning models to study the presence of post-storm cooling in the middle-thermosphere. We find that both NRLS 2.0 and JB2008-ML do not account for post-storm cooling and perform poorly in periods following strong geomagnetic storms. Results show that density reductions up to 40% can occur 1--3 days post-storm depending on location and the strength of the storm.
  arXiv  Detail & Related papers  (2022-06-12T19:40:30Z)
• Uncertainty estimation of pedestrian future trajectory using Bayesian approximation [137.00426219455116]
  Under dynamic traffic scenarios, planning based on deterministic predictions is not trustworthy. The authors propose to quantify uncertainty during forecasting using approximation which deterministic approaches fail to capture. The effect of dropout weights and long-term prediction on future state uncertainty has been studied.
  arXiv  Detail & Related papers  (2022-05-04T04:23:38Z)
• Dense Uncertainty Estimation [62.23555922631451]
  In this paper, we investigate neural networks and uncertainty estimation techniques to achieve both accurate deterministic prediction and reliable uncertainty estimation. We work on two types of uncertainty estimations solutions, namely ensemble based methods and generative model based methods, and explain their pros and cons while using them in fully/semi/weakly-supervised framework.
  arXiv  Detail & Related papers  (2021-10-13T01:23:48Z)
• Machine-Learned HASDM Model with Uncertainty Quantification [0.0]
  We develop the first thermospheric neutral mass density model with robust and reliable uncertainty estimates. We compare the best HASDM-ML model to the U.S. Space Force's High Accuracy Satellite Drag Model database. The model provides robust and reliable uncertainties in the density space over all space weather conditions.
  arXiv  Detail & Related papers  (2021-09-16T01:06:44Z)
• Improving Uncertainty Calibration via Prior Augmented Data [56.88185136509654]
  Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators. They are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions. We propose a solution by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels.
  arXiv  Detail & Related papers  (2021-02-22T07:02:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.