From ARIMA to Attention: Power Load Forecasting Using Temporal Deep Learning

• URL: http://arxiv.org/abs/2603.06622v1
• Date: Sat, 21 Feb 2026 10:51:49 GMT
• Title: From ARIMA to Attention: Power Load Forecasting Using Temporal Deep Learning
• Abstract summary: This paper performs an empirical evaluation of a traditional statistical model and deep learning approaches for predicting short-term energy load.<n>Four models, namely ARIMA, LSTM, BiLSTM, and Transformer, were leveraged on the PJM Hourly Energy Consumption data.<n>The Transformer model, which relies on self-attention algorithms, produced the best results with 3.8 percent of MAPE.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate short-term power load forecasting is important to effectively manage, optimize, and ensure the robustness of modern power systems. This paper performs an empirical evaluation of a traditional statistical model and deep learning approaches for predicting short-term energy load. Four models, namely ARIMA, LSTM, BiLSTM, and Transformer, were leveraged on the PJM Hourly Energy Consumption data. The data processing involved interpolation, normalization, and a sliding-window sequence method. Each model's forecasting performance was evaluated for the 24-hour horizon using MAE, RMSE, and MAPE. Of the models tested, the Transformer model, which relies on self-attention algorithms, produced the best results with 3.8 percent of MAPE, with performance above any model in both accuracy and robustness. These findings underscore the growing potential of attention-based architectures in accurately capturing complex temporal patterns in power consumption data.

Related papers

• An Empirical Benchmark of Deep Time-Series Models for Smart Meter Energy Forecasting [1.539942973115038]
  We present an empirical benchmark of nine modern deep learning models for time-series forecasting.<n>We evaluate these models on two publicly available smart meter datasets.
  arXiv  Detail & Related papers  (2026-08-19T08:27:48Z)
• Delta-Based Target Reformulation for Short-Term Electricity Load Forecasting Using LSTM and Transformer Models [0.0]
  This paper investigates a delta-based target reformulation for short-term electricity load forecasting using deep learning.<n>Instead of directly predicting absolute load values, the proposed formulation trains models to predict the change in load between consecutive time steps.<n>Results show that delta-based reformulation consistently improves forecasting accuracy for hour-ahead prediction.
  arXiv  Detail & Related papers  (2026-06-16T09:01:44Z)
• Forecasting Energy Consumption using Recurrent Neural Networks: A Comparative Analysis [0.0]
  We propose a forecasting approach based on Recurrent Neural Networks (RNNs) and their advanced variant, Long Short-Term Memory (LSTM) networks.<n>Our methodology integrates historical energy consumption data with external variables, including temperature, humidity, and time-based features.<n> Experimental results show that the LSTM model substantially outperforms the baseline, achieving lower Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE)
  arXiv  Detail & Related papers  (2026-01-23T18:14:53Z)
• Information Capacity: Evaluating the Efficiency of Large Language Models via Text Compression [53.39128997308138]
  We introduce information capacity, a measure of model efficiency based on text compression performance.<n> Empirical evaluations on mainstream open-source models show that models of varying sizes within a series exhibit consistent information capacity.<n>A distinctive feature of information capacity is that it incorporates tokenizer efficiency, which affects both input and output token counts.
  arXiv  Detail & Related papers  (2025-11-11T10:07:32Z)
• Time Series Forecasting Using a Hybrid Deep Learning Method: A Bi-LSTM Embedding Denoising Auto Encoder Transformer [0.0]
  This study introduces a BI-LSTM embedding denoising autoencoder model (BDM) designed to address time series problems.<n>The performance of the proposed model is evaluated by comparing it with benchmark models like Transformer, CNN, RNN, LSTM, and GRU.
  arXiv  Detail & Related papers  (2025-09-21T17:16:32Z)
• A Hybrid PCA-PR-Seq2Seq-Adam-LSTM Framework for Time-Series Power Outage Prediction [5.657115189763182]
  This paper introduces a hybrid deep learning framework, termed PCA-PR-Seq2Seq-Adam-LSTM.<n>It integrates Principal Component Analysis (PCA), Poisson Regression (PR), a Sequence-to-Sequence (Seq2Seq) architecture, and an Adam-optimized LSTM.<n>Results indicate that the proposed approach significantly improves forecasting accuracy and robustness compared to existing methods.
  arXiv  Detail & Related papers  (2025-09-20T17:13:25Z)
• Efficient Federated Learning with Timely Update Dissemination [54.668309196009204]
  Federated Learning (FL) has emerged as a compelling methodology for the management of distributed data.<n>We propose an efficient FL approach that capitalizes on additional downlink bandwidth resources to ensure timely update dissemination.
  arXiv  Detail & Related papers  (2025-07-08T14:34:32Z)
• Probabilistic Forecasting for Building Energy Systems using Time-Series Foundation Models [20.57107693396709]
  Building energy systems critically depend on the predictive accuracy of relevant time-series models.<n>This paper investigates the applicability and fine-tuning strategies of time-series foundation models (TSFMs) in building energy forecasting.
  arXiv  Detail & Related papers  (2025-05-31T16:38:29Z)
• EfficientLLM: Efficiency in Large Language Models [64.3537131208038]
  Large Language Models (LLMs) have driven significant progress, yet their growing counts and context windows incur prohibitive compute, energy, and monetary costs.<n>We introduce EfficientLLM, a novel benchmark and the first comprehensive empirical study evaluating efficiency techniques for LLMs at scale.
  arXiv  Detail & Related papers  (2025-05-20T02:27:08Z)
• Powerformer: A Transformer with Weighted Causal Attention for Time-series Forecasting [50.298817606660826]
  We introduce Powerformer, a novel Transformer variant that replaces noncausal attention weights with causal weights that are reweighted according to a smooth heavy-tailed decay.<n>Our empirical results demonstrate that Powerformer achieves state-of-the-art accuracy on public time-series benchmarks.<n>Our analyses show that the model's locality bias is amplified during training, demonstrating an interplay between time-series data and power-law-based attention.
  arXiv  Detail & Related papers  (2025-02-10T04:42:11Z)
• FengWu-4DVar: Coupling the Data-driven Weather Forecasting Model with 4D Variational Assimilation [67.20588721130623]
  We develop an AI-based cyclic weather forecasting system, FengWu-4DVar. FengWu-4DVar can incorporate observational data into the data-driven weather forecasting model. Experiments on the simulated observational dataset demonstrate that FengWu-4DVar is capable of generating reasonable analysis fields.
  arXiv  Detail & Related papers  (2023-12-16T02:07:56Z)
• A comparative assessment of deep learning models for day-ahead load forecasting: Investigating key accuracy drivers [2.572906392867547]
  Short-term load forecasting (STLF) is vital for the effective and economic operation of power grids and energy markets. Several deep learning models have been proposed in the literature for STLF, reporting promising results.
  arXiv  Detail & Related papers  (2023-02-23T17:11:04Z)
• Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution [57.71199089609161]
  Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. We propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects.
  arXiv  Detail & Related papers  (2023-01-05T13:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.