Ensemble Score Filtering for Real-Data Energy Consumption Forecast Correction

• URL: http://arxiv.org/abs/2605.29072v2
• Date: Mon, 01 Jun 2026 15:36:10 GMT
• Title: Ensemble Score Filtering for Real-Data Energy Consumption Forecast Correction
• Abstract summary: We study a high-dimensional data assimilation problem for real energy-consumption data.<n>We employ the Ensemble Score Filter (EnSF) to assimilate partial and noisy observations and correct the forecast trajectory over time.
• Score: 11.027420128235903
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate estimation and forecasting of energy consumption are important for power-system operation, planning, and demand-side management. In practice, however, complete and timely measurements may not always be available, and the observed data can be partial, noisy, or delayed. This motivates the use of learned forecasting models for predicting the evolving consumption state, together with data assimilation methods for sequential forecast correction. In this work, we study a high-dimensional data assimilation problem for real energy-consumption data. \modeltext{The forward prediction is supplied by a pretrained black-box spatio-temporal forecasting model, which is treated as the state propagator in the filtering procedure.} We employ the Ensemble Score Filter (EnSF) to assimilate partial and noisy observations and to correct the forecast trajectory over time. The EnSF uses score-based diffusion models to approximate filtering distributions and avoids retraining neural-network score models during assimilation by using a closed-form score representation and Monte Carlo approximation. Numerical experiments demonstrate that open-loop propagation of the learned forecasting model can become unreliable over long horizons, while EnSF-based correction substantially improves state estimation. Comparisons with the Ensemble Kalman Filter (EnKF) further show that EnSF provides stronger correction under the nonlinear observation setting considered in this work.

Related papers

• DiFA: Inference-Time Forward-Process Alignment for Diffusion Models [8.208222349301785]
  Forward-Process Aligned Diffusion prediction (textbfDiFA) is a training-free framework that reframes inference-time data prediction refinement as a sequential state estimation problem.<n>Inspired by Kalman filtering, DiFA aggregates historical predictions according to structural consistency and noise-level compatibility.
  arXiv  Detail & Related papers  (2026-07-20T14:10:53Z)
• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• Inpainting-Style Conditional Diffusion for Multivariable Time Series Forecasting [15.992262494632321]
  We propose a conditional diffusion-based framework for multitemporal time-series solar forecasting power.<n>The proposed method reformulates temporal PV data as structured two-dimensional representations using a sliding-window patch construction.<n>Experiments conducted on benchmark PV dataset, including GEFCom2014, demonstrate that the approach achieves high forecasting accuracy.
  arXiv  Detail & Related papers  (2026-05-27T11:25:28Z)
• Prequential posteriors [2.831395148295604]
  We introduce prequential posteriors, based upon a predictive-sequential (prequential) loss function.<n>We prove that, under mild conditions, both the prequential loss minimizer and the prequential posterior concentrate around parameters with optimal predictive performance.<n>We validate our method on both a synthetic multi-dimensional time series and a real-world meteorological dataset.
  arXiv  Detail & Related papers  (2025-11-21T19:18:19Z)
• DistDF: Time-Series Forecasting Needs Joint-Distribution Wasserstein Alignment [92.70019102733453]
  Training time-series forecast models requires aligning the conditional distribution of model forecasts with that of the label sequence.<n>We propose DistDF, which achieves alignment by alternatively minimizing a discrepancy between the conditional forecast and label distributions.
  arXiv  Detail & Related papers  (2025-10-28T16:09:59Z)
• The Living Forecast: Evolving Day-Ahead Predictions into Intraday Reality [0.688204255655161]
  This paper introduces a Bayesian updating mechanism that converts fully probabilistic day-ahead forecasts into intraday forecasts without retraining or re-inference.<n>Experiments on household electricity consumption and photovoltaic generation datasets demonstrate that the proposed method improves forecast accuracy up to 25% across likelihood-, sample-, quantile-, and point-based metrics.
  arXiv  Detail & Related papers  (2025-10-14T08:25:20Z)
• Bridging the Last Mile of Prediction: Enhancing Time Series Forecasting with Conditional Guided Flow Matching [9.465542901469815]
  Conditional Guided Flow Matching (CGFM) is a model-agnostic framework that extends flow matching by integrating outputs from an auxiliary predictive model.<n>CGFM incorporates historical data as both conditions and guidance, uses two-sided conditional paths, and employs affine paths to expand the path space.<n> Experiments across datasets and baselines show CGFM consistently outperforms state-of-the-art models, advancing forecasting.
  arXiv  Detail & Related papers  (2025-07-09T18:03:31Z)
• On conditional diffusion models for PDE simulations [53.01911265639582]
  We study score-based diffusion models for forecasting and assimilation of sparse observations. We propose an autoregressive sampling approach that significantly improves performance in forecasting. We also propose a new training strategy for conditional score-based models that achieves stable performance over a range of history lengths.
  arXiv  Detail & Related papers  (2024-10-21T18:31:04Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• DiffDA: a Diffusion Model for Weather-scale Data Assimilation [19.336483240566142]
  We propose DiffDA as a denoising diffusion model capable of assimilating atmospheric variables using predicted states and sparse observations. Acknowledging the similarity between a weather forecast model and a denoising diffusion model dedicated to weather applications, we adapt the pretrained GraphCast neural network as the backbone of the diffusion model.
  arXiv  Detail & Related papers  (2024-01-11T14:11:12Z)
• Prediction-Oriented Bayesian Active Learning [51.426960808684655]
  Expected predictive information gain (EPIG) is an acquisition function that measures information gain in the space of predictions rather than parameters. EPIG leads to stronger predictive performance compared with BALD across a range of datasets and models.
  arXiv  Detail & Related papers  (2023-04-17T10:59:57Z)
• DeepVol: Volatility Forecasting from High-Frequency Data with Dilated Causal Convolutions [53.37679435230207]
  We propose DeepVol, a model based on Dilated Causal Convolutions that uses high-frequency data to forecast day-ahead volatility. Our empirical results suggest that the proposed deep learning-based approach effectively learns global features from high-frequency data.
  arXiv  Detail & Related papers  (2022-09-23T16:13:47Z)
• Back2Future: Leveraging Backfill Dynamics for Improving Real-time Predictions in Future [73.03458424369657]
  In real-time forecasting in public health, data collection is a non-trivial and demanding task. 'Backfill' phenomenon and its effect on model performance has been barely studied in the prior literature. We formulate a novel problem and neural framework Back2Future that aims to refine a given model's predictions in real-time.
  arXiv  Detail & Related papers  (2021-06-08T14:48:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.