Robust Calibration For Improved Weather Prediction Under Distributional Shift

• URL: http://arxiv.org/abs/2401.04144v1
• Date: Mon, 8 Jan 2024 01:29:56 GMT
• Title: Robust Calibration For Improved Weather Prediction Under Distributional Shift
• Authors: Sankalp Gilda, Neel Bhandari, Wendy Mak, Andrea Panizza
• Abstract summary: We present results on improving out-of-domain weather prediction and uncertainty estimation as part of the textttShifts Challenge on Robustness and Uncertainty. We find that by leveraging a mixture of experts in conjunction with an advanced data augmentation technique borrowed from the computer vision domain, in conjunction with robust textitpost-hoc calibration of predictive uncertainties, we can potentially achieve more accurate and better-calibrated results.
• Score: 3.6248657646376707
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In this paper, we present results on improving out-of-domain weather prediction and uncertainty estimation as part of the \texttt{Shifts Challenge on Robustness and Uncertainty under Real-World Distributional Shift} challenge. We find that by leveraging a mixture of experts in conjunction with an advanced data augmentation technique borrowed from the computer vision domain, in conjunction with robust \textit{post-hoc} calibration of predictive uncertainties, we can potentially achieve more accurate and better-calibrated results with deep neural networks than with boosted tree models for tabular data. We quantify our predictions using several metrics and propose several future lines of inquiry and experimentation to boost performance.

Related papers

• Diffusion-based Time Series Forecasting for Sewerage Systems [0.3495246564946556]
  We introduce a novel deep learning approach to enhance the accuracy of contextual forecasting in sewerage systems.<n>Our system excels at capturing complex correlations across diverse environmental signals, enabling robust predictions even during extreme weather events.<n>Our empirical tests on real sewerage system data confirm the model's exceptional capability to deliver reliable contextual predictions.
  arXiv  Detail & Related papers  (2025-06-10T08:48:05Z)
• Calibrated Probabilistic Forecasts for Arbitrary Sequences [58.54729945445505]
  Real-world data streams can change unpredictably due to distribution shifts, feedback loops and adversarial actors. We present a forecasting framework ensuring valid uncertainty estimates regardless of how data evolves.
  arXiv  Detail & Related papers  (2024-09-27T21:46:42Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• Performative Time-Series Forecasting [71.18553214204978]
  We formalize performative time-series forecasting (PeTS) from a machine-learning perspective. We propose a novel approach, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts. We conduct comprehensive experiments using multiple time-series models on COVID-19 and traffic forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-09T18:34:29Z)
• Multiclass Alignment of Confidence and Certainty for Network Calibration [10.15706847741555]
  Recent studies reveal that deep neural networks (DNNs) are prone to making overconfident predictions. We propose a new train-time calibration method, which features a simple, plug-and-play auxiliary loss known as multi-class alignment of predictive mean confidence and predictive certainty (MACC) Our method achieves state-of-the-art calibration performance for both in-domain and out-domain predictions.
  arXiv  Detail & Related papers  (2023-09-06T00:56:24Z)
• PreDiff: Precipitation Nowcasting with Latent Diffusion Models [28.52267957954304]
  We develop a conditional latent diffusion model capable of probabilistic forecasts. We incorporate an explicit knowledge alignment mechanism to align forecasts with domain-specific physical constraints.
  arXiv  Detail & Related papers  (2023-07-19T19:19:13Z)
• Learning Sample Difficulty from Pre-trained Models for Reliable Prediction [55.77136037458667]
  We propose to utilize large-scale pre-trained models to guide downstream model training with sample difficulty-aware entropy regularization. We simultaneously improve accuracy and uncertainty calibration across challenging benchmarks.
  arXiv  Detail & Related papers  (2023-04-20T07:29:23Z)
• A Bayesian Deep Learning Approach to Near-Term Climate Prediction [12.870804083819603]
  We pursue a complementary machine-learning-based approach to climate prediction. In particular, we find that a feedforward convolutional network with a Densenet architecture is able to outperform a convolutional LSTM in terms of predictive skill.
  arXiv  Detail & Related papers  (2022-02-23T00:28:36Z)
• Probabilistic Deep Learning to Quantify Uncertainty in Air Quality Forecasting [5.007231239800297]
  This work applies state-of-the-art techniques of uncertainty quantification in a real-world setting of air quality forecasts. We describe training probabilistic models and evaluate their predictive uncertainties based on empirical performance, reliability of confidence estimate, and practical applicability. Our experiments demonstrate that the proposed models perform better than previous works in quantifying uncertainty in data-driven air quality forecasts.
  arXiv  Detail & Related papers  (2021-12-05T17:01:18Z)
• Probabilistic electric load forecasting through Bayesian Mixture Density Networks [70.50488907591463]
  Probabilistic load forecasting (PLF) is a key component in the extended tool-chain required for efficient management of smart energy grids. We propose a novel PLF approach, framed on Bayesian Mixture Density Networks. To achieve reliable and computationally scalable estimators of the posterior distributions, both Mean Field variational inference and deep ensembles are integrated.
  arXiv  Detail & Related papers  (2020-12-23T16:21:34Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.