Advancing Heat Demand Forecasting with Attention Mechanisms: Opportunities and Challenges

• URL: http://arxiv.org/abs/2502.07854v1
• Date: Tue, 11 Feb 2025 13:12:06 GMT
• Title: Advancing Heat Demand Forecasting with Attention Mechanisms: Opportunities and Challenges
• Abstract summary: We build a Deep Learning (DL) model that uses deconstructed components of independent and dependent variables that affect heat demand as features to perform multi-step ahead forecasting of head demand.<n>The proposed method is evaluated on a real-world dataset and the forecasting performance is assessed against LSTM and CNN-based forecasting models.
• Score: 2.979183050755201
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Global leaders and policymakers are unified in their unequivocal commitment to decarbonization efforts in support of Net-Zero agreements. District Heating Systems (DHS), while contributing to carbon emissions due to the continued reliance on fossil fuels for heat production, are embracing more sustainable practices albeit with some sense of vulnerability as it could constrain their ability to adapt to dynamic demand and production scenarios. As demographic demands grow and renewables become the central strategy in decarbonizing the heating sector, the need for accurate demand forecasting has intensified. Advances in digitization have paved the way for Machine Learning (ML) based solutions to become the industry standard for modeling complex time series patterns. In this paper, we focus on building a Deep Learning (DL) model that uses deconstructed components of independent and dependent variables that affect heat demand as features to perform multi-step ahead forecasting of head demand. The model represents the input features in a time-frequency space and uses an attention mechanism to generate accurate forecasts. The proposed method is evaluated on a real-world dataset and the forecasting performance is assessed against LSTM and CNN-based forecasting models. Across different supply zones, the attention-based models outperforms the baselines quantitatively and qualitatively, with an Mean Absolute Error (MAE) of 0.105 with a standard deviation of 0.06kW h and a Mean Absolute Percentage Error (MAPE) of 5.4% with a standard deviation of 2.8%, in comparison the second best model with a MAE of 0.10 with a standard deviation of 0.06kW h and a MAPE of 5.6% with a standard deviation of 3%.

Related papers

• IPM-FM: A Foundation Model with Consensus Feature Selection for Industrial Process Monitoring [59.59850511287819]
  We propose the industrial process monitoring foundation model (IPM-FM)<n>It first learns general-purpose representations from unlabeled industrial process data through self-supervised pretraining.<n>It adapts to specific monitoring tasks using a small amount of task-labeled data, and finally produces calibrated predictions.
  arXiv  Detail & Related papers  (2026-09-08T07:43:04Z)
• Federated Learning for Global Carbon Emission Forecasting: A Hybrid Time-Series Approach with Statistical and Neural Models [5.43501294838216]
  This paper proposes a novel hybrid forecasting framework that integrates ARIMA-based trend modeling, GARCH-based volatility modeling, LSTM-Attention temporal representation learning, and XGBoost prediction.<n>The proposed framework enables collaborative learning among distributed clients without requiring the exchange of raw data.<n>The results indicate that the proposed framework provides an accurate, scalable, and regulation-compliant solution for collaborative carbon-emission forecasting.
  arXiv  Detail & Related papers  (2026-06-21T17:58:53Z)
• Time series Foundation Models based on Physics-Informed Synthetic Histories for Cold-Start Photovoltaic Forecasting [45.753923525593045]
  The evaluation spans $440$ PV sites across four datasets and diverse climate regimes.<n>Performance is largely insensitive to the synthetic-history source, indicating that accuracy is driven more by the availability of plausible temporal context than by the specific generator.
  arXiv  Detail & Related papers  (2026-06-05T17:05:19Z)
• Regression Models Meet Foundation Models: A Hybrid-AI Approach to Practical Electricity Price Forecasting [39.91788164419845]
  We propose FutureBoosting, a novel paradigm that enhances regression-based forecasts.<n>We instantiate this paradigm into a lightweight, plug-and-play framework for electricity price forecasting.<n>Our framework consistently outperforms state-of-the-art TSFMs and regression baselines.
  arXiv  Detail & Related papers  (2026-03-06T02:06:46Z)
• China Regional 3km Downscaling Based on Residual Corrective Diffusion Model [39.12803910865843]
  This work focuses on statistical downscaling, which establishes statistical relationships between low-resolution and high-resolution historical data.<n>In contrast to the original work of CorrDiff, the region considered in this work is nearly 40 times larger.<n>Deep learning has emerged as a powerful tool for this task, giving rise to various high-performance super-resolution models.
  arXiv  Detail & Related papers  (2025-12-05T02:27:08Z)
• Data-driven jet fuel demand forecasting: A case study of Copenhagen Airport [43.17090130312271]
  We evaluate the performance of data-driven approaches using a substantial amount of data obtained from a major aviation fuel distributor in the Danish market.<n>Our analysis compares the predictive capabilities of traditional time series models, Prophet, LSTM sequence-to-sequence neural networks, and hybrid models.<n>To ensure the reliability of the data-driven approaches and provide valuable insights to practitioners, we analyze three different datasets.
  arXiv  Detail & Related papers  (2025-11-04T11:45:58Z)
• Km-scale dynamical downscaling through conformalized latent diffusion models [45.94979929172337]
  Dynamical downscaling is crucial for deriving high-resolution meteorological fields from coarse-scale simulations.<n>Generative Diffusion models (DMs) have recently emerged as powerful data-driven tools for this task.<n>However, DMs lack finite-sample guarantees against overconfident predictions, resulting in miscalibrated grid-point-level uncertainty estimates.<n>We tackle this issue by augmenting the downscaling pipeline with a conformal prediction framework.
  arXiv  Detail & Related papers  (2025-10-15T08:41:36Z)
• Partially stochastic deep learning with uncertainty quantification for model predictive heating control [0.0]
  Model Predictive Control (MPC) addresses drawbacks by using predictive modeling to optimize heating based on a building's learned thermal behavior.<n>MPC solutions often employ simplified physics-inspired indoor temperature models, sacrificing accuracy for robustness and interpretability.<n>We propose a partially deep learning architecture, dubbed LSTM+BNN, for building-specific indoor temperature modeling.
  arXiv  Detail & Related papers  (2025-04-04T11:07:23Z)
• The Lessons of Developing Process Reward Models in Mathematical Reasoning [62.165534879284735]
  Process Reward Models (PRMs) aim to identify and mitigate intermediate errors in the reasoning processes.<n>We develop a consensus filtering mechanism that effectively integrates Monte Carlo (MC) estimation with Large Language Models (LLMs)<n>We release a new state-of-the-art PRM that outperforms existing open-source alternatives.
  arXiv  Detail & Related papers  (2025-01-13T13:10:16Z)
• On conditional diffusion models for PDE simulations [53.01911265639582]
  We study score-based diffusion models for forecasting and assimilation of sparse observations. We propose an autoregressive sampling approach that significantly improves performance in forecasting. We also propose a new training strategy for conditional score-based models that achieves stable performance over a range of history lengths.
  arXiv  Detail & Related papers  (2024-10-21T18:31:04Z)
• COME: Test-time adaption by Conservatively Minimizing Entropy [45.689829178140634]
  Conservatively Minimize the Entropy (COME) is a drop-in replacement of traditional entropy (EM) COME explicitly models the uncertainty by characterizing a Dirichlet prior distribution over model predictions. We show that COME achieves state-of-the-art performance on commonly used benchmarks.
  arXiv  Detail & Related papers  (2024-10-12T09:20:06Z)
• Towards Universal Large-Scale Foundational Model for Natural Gas Demand Forecasting [12.60741035434783]
  We propose the first foundation model specifically tailored for natural gas demand forecasting. Our approach leverages contrastive learning to improve prediction accuracy in real-world scenarios. We conducted extensive experiments using a large-scale dataset from ENN Group.
  arXiv  Detail & Related papers  (2024-09-24T06:44:29Z)
• F-FOMAML: GNN-Enhanced Meta-Learning for Peak Period Demand Forecasting with Proxy Data [65.6499834212641]
  We formulate the demand prediction as a meta-learning problem and develop the Feature-based First-Order Model-Agnostic Meta-Learning (F-FOMAML) algorithm. By considering domain similarities through task-specific metadata, our model improved generalization, where the excess risk decreases as the number of training tasks increases. Compared to existing state-of-the-art models, our method demonstrates a notable improvement in demand prediction accuracy, reducing the Mean Absolute Error by 26.24% on an internal vending machine dataset and by 1.04% on the publicly accessible JD.com dataset.
  arXiv  Detail & Related papers  (2024-06-23T21:28:50Z)
• FuXi-ENS: A machine learning model for medium-range ensemble weather forecasting [16.562512279873577]
  We introduce FuXi-ENS, an advanced ML model designed to deliver 6-hourly global ensemble weather forecasts up to 15 days. FuXi-ENS runs at a significantly increased spatial resolution of 0.25textdegree, incorporating 5 atmospheric variables at 13 pressure levels, along with 13 surface variables. Results demonstrate that FuXi-ENS outperforms ensemble forecasts from the ECMWF, a world leading NWP model, in the CRPS of 98.1% of 360 variable and forecast lead time combinations.
  arXiv  Detail & Related papers  (2024-05-09T17:15:09Z)
• Weather Prediction with Diffusion Guided by Realistic Forecast Processes [49.07556359513563]
  We introduce a novel method that applies diffusion models (DM) for weather forecasting. Our method can achieve both direct and iterative forecasting with the same modeling framework. The flexibility and controllability of our model empowers a more trustworthy DL system for the general weather community.
  arXiv  Detail & Related papers  (2024-02-06T21:28:42Z)
• Long-term drought prediction using deep neural networks based on geospatial weather data [75.38539438000072]
  High-quality drought forecasting up to a year in advance is critical for agriculture planning and insurance. We tackle drought data by introducing an end-to-end approach that adopts a systematic end-to-end approach. Key findings are the exceptional performance of a Transformer model, EarthFormer, in making accurate short-term (up to six months) forecasts.
  arXiv  Detail & Related papers  (2023-09-12T13:28:06Z)
• Cal-SFDA: Source-Free Domain-adaptive Semantic Segmentation with Differentiable Expected Calibration Error [50.86671887712424]
  The prevalence of domain adaptive semantic segmentation has prompted concerns regarding source domain data leakage. To circumvent the requirement for source data, source-free domain adaptation has emerged as a viable solution. We propose a novel calibration-guided source-free domain adaptive semantic segmentation framework.
  arXiv  Detail & Related papers  (2023-08-06T03:28:34Z)
• A generalised multi-factor deep learning electricity load forecasting model for wildfire-prone areas [0.0]
  This paper proposes a generalised and robust multi-factor Gated Recurrent Unit (GRU) based Deep Learning (DL) model to forecast electricity load in distribution networks during wildfire seasons. Compared to the regular use of instantaneous temperature, the Mean Absolute Percentage Error (MAPE) is decreased by 30.73%. The proposed method achieves an average forecast MAPE of around 3%, giving a potential annual energy saving of AU$80.46 million for the state of Victoria.
  arXiv  Detail & Related papers  (2023-04-21T00:42:24Z)
• A comparative study of statistical and machine learning models on near-real-time daily emissions prediction [0.0]
  The rapid ascent in carbon dioxide emissions is a major cause of global warming and climate change. This paper aims to select a suitable model to predict the near-real-time daily emissions from January 1st, 2020 to September 30st, 2022 of all sectors in China.
  arXiv  Detail & Related papers  (2023-02-02T15:14:27Z)
• A Hybrid Model for Forecasting Short-Term Electricity Demand [59.372588316558826]
  Currently the UK Electric market is guided by load (demand) forecasts published every thirty minutes by the regulator. We present HYENA: a hybrid predictive model that combines feature engineering (selection of the candidate predictor features), mobile-window predictors and LSTM encoder-decoders.
  arXiv  Detail & Related papers  (2022-05-20T22:13:25Z)
• LIMREF: Local Interpretable Model Agnostic Rule-based Explanations for Forecasting, with an Application to Electricity Smart Meter Data [3.0839245814393728]
  We propose Local Interpretable Model-agnostic Rule-based Explanations for Forecasting (LIMREF) to explain global model forecasts. Here, we evaluate the quality of the explanations produced by the LIMREF framework in terms of both qualitative and quantitative aspects.
  arXiv  Detail & Related papers  (2022-02-15T22:35:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.