Benchmarking Transformer and xLSTM for Time-Series Forecasting of Heat Consumption

• URL: http://arxiv.org/abs/2605.09722v1
• Date: Sun, 10 May 2026 19:40:03 GMT
• Title: Benchmarking Transformer and xLSTM for Time-Series Forecasting of Heat Consumption
• Abstract summary: This paper benchmarks Transformer-based and xLSTM architectures for short-term heat-demand forecasting.<n>We establish a benchmark that tests whether models trained on pooled, heterogeneous building data are able to generalize across diverse building stock.<n>The results show that the xLSTM achieves the lowest RMSE (19.88 kWh for three-hour, 21.47 kWh for 24-hour forecasts), while the Temporal Fusion Transformer attains the best MAE (9.16 kWh for three-hour forecasts)
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Obtaining an accurate short-term forecasting for heat demand is an essential part of operating district heating networks cost-efficient and reliable. Heat consumption time series at the building level are highly dependent on exogenous variables such as outdoor temperature and individual usage patterns, making forecasting in this context a challenging task. Thus, this paper benchmarks novel Transformer-based and xLSTM architectures for short-term heat-demand forecasting. Using hourly data from 25 German buildings (2017-2025), we compare three-hour and 24-hour forecasting horizons relevant for intraday control and day-ahead scheduling. We establish a multi-building benchmark that tests whether models trained on pooled, heterogeneous building data are able to generalize across diverse building stock. The results show that the xLSTM achieves the lowest RMSE (19.88 kWh for three-hour, 21.47 kWh for 24-hour forecasts), while the Temporal Fusion Transformer attains the best MAE (9.16 kWh for three-hour forecasts). As xLSTMs and Transformers require long training times and have a huge number of trainable parameters, their sustainability remains questionable. Therefore, this paper further investigates the trade-off between predictive accuracy and computational resource demand of the evaluated forecasting models. The findings indicate that also low-parameter models like a traditional fully-connected network achieve good predictive results, highlighting that marginal accuracy gains of the novel prediction models come at substantial resource expense for this use case.

Related papers

• Systematic Evaluation of TabPFN-TS for Zero-Shot Probabilistic Heat Load Forecasting in District Heating Networks [35.66975292316149]
  District energy hubs require reliable heat load forecasts for efficient operational scheduling.<n>Time-series foundation models and in-context forecasting offer a promising alternative.<n>This study systematically evaluates TabPFN-TS against time-series foundation models and trained machine-learning baselines.
  arXiv  Detail & Related papers  (2026-08-20T13:33:37Z)
• Time series Foundation Models based on Physics-Informed Synthetic Histories for Cold-Start Photovoltaic Forecasting [45.753923525593045]
  The evaluation spans $440$ PV sites across four datasets and diverse climate regimes.<n>Performance is largely insensitive to the synthetic-history source, indicating that accuracy is driven more by the availability of plausible temporal context than by the specific generator.
  arXiv  Detail & Related papers  (2026-06-05T17:05:19Z)
• Time-Aware Prior Fitted Networks for Zero-Shot Forecasting with Exogenous Variables [59.37476887007663]
  We develop ApolloPFN, a prior-data fitted network (PFN) that is time-aware (unlike prior PFNs)<n>Our design introduces two major advances: (i) a synthetic data generation procedure tailored to resolve the failure modes that arise when PFNs are applied to time series; and (ii) time-aware architectural modifications that embed inductive biases needed to exploit the time series context.
  arXiv  Detail & Related papers  (2026-03-16T18:33:23Z)
• A Deep Learning Framework for Heat Demand Forecasting using Time-Frequency Representations of Decomposed Features [3.0294044944521694]
  We propose a novel deep learning framework for day-ahead heat demand prediction.<n>By applying Continuous Wavelet Transform to demand and external meteorological factors, our approach enables Convolutional Neural Networks to learn hierarchical temporal features.<n>The results show a significant advancement, reducing the Mean Absolute Error by 36% to 43% compared to the strongest baselines.
  arXiv  Detail & Related papers  (2026-03-01T14:40:13Z)
• Echo State Networks for Time Series Forecasting: Hyperparameter Sweep and Benchmarking [51.56484100374058]
  We evaluate whether a fully automatic, purely feedback-driven ESN can serve as a competitive alternative to widely used statistical forecasting methods.<n>Forecast accuracy is measured using MASE and sMAPE and benchmarked against simple benchmarks like drift and seasonal naive and statistical models.
  arXiv  Detail & Related papers  (2026-02-03T16:01:22Z)
• Efficient Deep Learning for Short-Term Solar Irradiance Time Series Forecasting: A Benchmark Study in Ho Chi Minh City [0.0]
  This study presents a benchmark of ten deep learning architectures for short-term (1-hour ahead) Global Horizontal Irradiance (GHI) forecasting in Ho Chi Minh City.<n> Experimental results identify the Transformer as the superior architecture, achieving the highest predictive accuracy with an R2 of 0.9696.
  arXiv  Detail & Related papers  (2025-12-29T23:22:25Z)
• SEMPO: Lightweight Foundation Models for Time Series Forecasting [45.456949943052116]
  SEMPO is a lightweight foundation model that requires pretraining on relatively small-scale data, yet exhibits strong general time series forecasting.<n> SEMPO comprises two key modules: 1) energy-aware SpEctral decomposition module, that substantially improves the utilization of pre-training data.<n>Experiments on two large-scale benchmarks covering 16 datasets demonstrate the superior performance of SEMPO in both zero-shot and few-shot forecasting scenarios.
  arXiv  Detail & Related papers  (2025-10-22T15:58:44Z)
• A Lightweight DL Model for Smart Grid Power Forecasting with Feature and Resolution Mismatch [0.4999814847776097]
  This paper challenges teams to predict-day power demand using real-world high-frequency data.<n>We propose a robust yet lightweight Deep Learning pipeline combining hourly downsizing, dual-mode imputation, and comprehensive normalization.<n>A sequence-to-one model achieves an average RMSE of 601.9W, MAE of 468.9W, and 84.36% accuracy.
  arXiv  Detail & Related papers  (2025-10-19T16:12:53Z)
• IISE PG&E Energy Analytics Challenge 2025: Hourly-Binned Regression Models Beat Transformers in Load Forecasting [0.0]
  This study evaluates forecasting models ranging from classical regression techniques to advanced deep learning architectures.<n>The dataset includes two years of historical electricity load data, alongside temperature and global horizontal irradiance (GHI) across five sites.<n>Our results reveal that deep learning models, including TimeGPT, fail to consistently outperform simpler statistical and machine learning approaches.
  arXiv  Detail & Related papers  (2025-05-16T15:55:34Z)
• Benchmarks and Custom Package for Energy Forecasting [55.460452605056894]
  Energy forecasting aims to minimize the cost of subsequent tasks such as power grid dispatch. In this paper, we collected large-scale load datasets and released a new renewable energy dataset. We conducted extensive experiments with 21 forecasting methods in these energy datasets at different levels under 11 evaluation metrics.
  arXiv  Detail & Related papers  (2023-07-14T06:50:02Z)
• Electricity Demand Forecasting with Hybrid Statistical and Machine Learning Algorithms: Case Study of Ukraine [0.0]
  The proposed methodology was constructed using hourly data from Ukraine's electricity consumption ranging from 2013 to 2020. Our hybrid model is very effective at forecasting long-term electricity consumption on an hourly resolution.
  arXiv  Detail & Related papers  (2023-04-11T12:15:50Z)
• Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution [57.71199089609161]
  Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. We propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects.
  arXiv  Detail & Related papers  (2023-01-05T13:59:29Z)
• A Hybrid Model for Forecasting Short-Term Electricity Demand [59.372588316558826]
  Currently the UK Electric market is guided by load (demand) forecasts published every thirty minutes by the regulator. We present HYENA: a hybrid predictive model that combines feature engineering (selection of the candidate predictor features), mobile-window predictors and LSTM encoder-decoders.
  arXiv  Detail & Related papers  (2022-05-20T22:13:25Z)
• FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators [45.520430157112884]
  FourCastNet accurately forecasts high-resolution, fast-timescale variables such as the surface wind speed, precipitation, and atmospheric water vapor. It has important implications for planning wind energy resources, predicting extreme weather events such as tropical cyclones, extra-tropical cyclones, and atmospheric rivers. FourCastNet generates a week-long forecast in less than 2 seconds, orders of magnitude faster than IFS.
  arXiv  Detail & Related papers  (2022-02-22T22:19:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.