Bayesian Transformer for Probabilistic Load Forecasting in Smart Grids

• URL: http://arxiv.org/abs/2603.07899v1
• Date: Mon, 09 Mar 2026 02:39:51 GMT
• Title: Bayesian Transformer for Probabilistic Load Forecasting in Smart Grids
• Abstract summary: This study proposes a Bayesian Transformer framework that integrates three complementary uncertainty mechanisms into a PatchTST backbone.<n>A seven-level multi-quantile pinball-loss prediction head and post-training isotonic regression calibration produce sharp, near-nominally covered prediction intervals.<n>On the primary benchmark (PJM, H=24h), BT achieves a CRPS of 0.0289, improving 7.4% over Deep Ensembles and 29.9% over the deterministic LSTM.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The reliable operation of modern power grids requires probabilistic load forecasts with well-calibrated uncertainty estimates. However, existing deep learning models produce overconfident point predictions that fail catastrophically under extreme weather distributional shifts. This study proposes a Bayesian Transformer (BT) framework that integrates three complementary uncertainty mechanisms into a PatchTST backbone: Monte Carlo Dropout for epistemic parameter uncertainty, variational feed-forward layers with log-uniform weight priors, and stochastic attention with learnable Gaussian noise perturbations on pre-softmax logits, representing, to the best of our knowledge, the first application of Bayesian attention to probabilistic load forecasting. A seven-level multi-quantile pinball-loss prediction head and post-training isotonic regression calibration produce sharp, near-nominally covered prediction intervals. Evaluation of five grid datasets (PJM, ERCOT, ENTSO-E Germany, France, and Great Britain) augmented with NOAA covariates across 24, 48, and 168-hour horizons demonstrates state-of-the-art performance. On the primary benchmark (PJM, H=24h), BT achieves a CRPS of 0.0289, improving 7.4% over Deep Ensembles and 29.9% over the deterministic LSTM, with 90.4% PICP at the 90% nominal level and the narrowest prediction intervals (4,960 MW) among all probabilistic baselines. During heat-wave and cold snap events, BT maintained 89.6% and 90.1% PICP respectively, versus 64.7% and 67.2% for the deterministic LSTM, confirming that Bayesian epistemic uncertainty naturally widens intervals for out-of-distribution inputs. Calibration remained stable across all horizons (89.8-90.4% PICP), while ablation confirmed that each component contributed a distinct value. The calibrated outputs directly support risk-based reserve sizing, stochastic unit commitment, and demand response activation.

Related papers

• Rollcast: Proper-Score Gated Rolling Anchors for Adaptive Probabilistic Time-Series Forecasting [0.0]
  Rolling means, medians, extrema, regression endpoints, and quantiles define candidate forecast locations.<n>A state-dependent softmax gate learns anchor probabilities by minimizing negative log predictive density.<n>Recursive simulation propagates the resulting mixture over multiple forecast horizons.
  arXiv  Detail & Related papers  (2026-09-03T18:50:29Z)
• SPARC: Single-Pass Scaling for Motion Forecasting with Conformal Bayesian Last Layers [1.4452289368758382]
  Single-Pass Adaptive Risk is a structured, calibrated, and efficient uncertainty estimate for human motion forecasters.<n>A deterministic backbone predicts the future mean, and a conjugate Bayesian last layer converts time-domain feature leverage into an analytic horizon-wise scale.<n>It ranks first on NLL and on the combined MPJPE+NLL criterion while retaining competitive point accuracy and efficient calibrated tubes.
  arXiv  Detail & Related papers  (2026-08-21T07:20:22Z)
• Tyche: One Step Flow for Efficient Probabilistic Weather Forecasting [48.263481135566316]
  We present Tyche, a one-step conditional flow model for efficient probabilistic weather forecasting.<n>Experiments on ERA5 at 1.5$circ$ and 6-hour resolution show that our Tyche, using merely a single NFE, matches or exceeds the forecast skill and calibration of state-of-the-art multi-step generative baselines.
  arXiv  Detail & Related papers  (2026-05-07T20:25:28Z)
• Bi-Level Chaotic Fusion Based Graph Convolutional Network for Stock Market Prediction Interval [0.0]
  Financial market forecasting is inherently uncertain, yet most deep learning approaches rely on single estimates without uncertainty.<n>The problem can be solved using prediction intervals, which allow obtaining an uncertainty and lower bound for the prediction.<n>Yet, the current methods tend to disregard relationships between assets or cannot simultaneously ensure good calibration sharpness.
  arXiv  Detail & Related papers  (2026-05-05T05:06:42Z)
• Activation Outliers in Transformer Quantization: Reproduction, Statistical Analysis, and Deployment Tradeoffs [0.0]
  Post-training quantization (PTQ) of transformers is known to suffer from severe accuracy degradation due to structured activation outliers.<n>This paper provides a reproducible empirical reproduction and systems-level extension of that phenomenon in BERT-base fine-tuned on QNLI.
  arXiv  Detail & Related papers  (2026-03-04T17:26:29Z)
• Variational Inference for Bayesian MIDAS Regression [0.0]
  We develop a Coordinate Ascent Variational Inference (CAVI) algorithm for regression with linear weight parameteri zations.<n>CAVI produces posterior means nearly identical to a block Gibbs sampler benchmark while achieving speedups of 107x to 1,772x.<n>Weight function parameters maintain excellent calibration (coverage above 92%) across all con gurations.
  arXiv  Detail & Related papers  (2026-02-23T08:51:26Z)
• Uncertainty-Aware Deep Learning Framework for Remaining Useful Life Prediction in Turbofan Engines with Learned Aleatoric Uncertainty [0.0]
  This research introduces a novel uncertainty-aware deep learning framework that learns aleatoric uncertainty directly through probabilistic modeling.<n>Our framework achieves breakthrough critical zone performance (RUL = 30 cycles) with RMSE of 5.14, 6.89, 5.27, and 7.16.<n>The learned uncertainty provides well-calibrated 95 percent confidence intervals with coverage ranging from 93.5 percent to 95.2 percent, enabling risk-aware maintenance scheduling.
  arXiv  Detail & Related papers  (2025-11-24T13:53:31Z)
• Bridging the Gap Between Bayesian Deep Learning and Ensemble Weather Forecasts [100.26854618129039]
  Weather forecasting is fundamentally challenged by the chaotic nature of the atmosphere.<n>Recent advances in Bayesian Deep Learning (BDL) offer a promising but often disconnected alternative.<n>We bridge these paradigms through a unified hybrid BDL framework for ensemble weather forecasting.
  arXiv  Detail & Related papers  (2025-11-18T07:49:52Z)
• Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoning [70.56067503630486]
  We argue that sixth-generation (6G) intelligence is not fluent token prediction but calibrated the capacity to imagine and choose.<n>We show that WM-MS3M cuts mean absolute error (MAE) by 1.69% versus MS3M with 32% fewer parameters and similar latency, and achieves 35-80% lower root mean squared error (RMSE) than attention/hybrid baselines with 2.3-4.1x faster inference.
  arXiv  Detail & Related papers  (2025-11-04T17:22:22Z)
• SPADE-S: A Sparsity-Robust Foundational Forecaster [31.818058575031888]
  SPADE-S is a robust forecasting architecture that significantly reduces magnitude- and sparsity-based systematic biases and improves overall prediction accuracy.<n>We show that SPADE-S can improve forecast accuracy by up to 15% depending on the quantile forecast and magnitude of the series.
  arXiv  Detail & Related papers  (2025-07-24T19:30:24Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2023-10-17T20:30:16Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2022-06-16T06:13:53Z)
• Regularization methods for the short-term forecasting of the Italian electric load [77.34726150561087]
  The problem of forecasting the whole 24 profile of the Italian electric load is addressed as a multitask learning problem. The 96x96 matrix weights form a 96x96 matrix, that can be seen and displayed as a surface sampled on a square domain. Different regularization and sparsity approaches to reduce the degrees of freedom of the surface were explored, comparing the obtained forecasts with those of the Italian Transmission System Operator Terna.
  arXiv  Detail & Related papers  (2021-12-08T22:15:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.