A Unifying Framework of Attention-based Neural Load Forecasting
- URL: http://arxiv.org/abs/2305.05082v1
- Date: Mon, 8 May 2023 22:46:54 GMT
- Title: A Unifying Framework of Attention-based Neural Load Forecasting
- Authors: Jing Xiong and Yu Zhang
- Abstract summary: We propose a unifying deep learning framework for load forecasting.
It includes time-varying feature weighting, hierarchical temporal attention, and feature-reinforced error correction.
Our framework provides an effective solution to the electric load forecasting problem.
- Score: 6.470432799969585
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Accurate load forecasting is critical for reliable and efficient planning and
operation of electric power grids. In this paper, we propose a unifying deep
learning framework for load forecasting, which includes time-varying feature
weighting, hierarchical temporal attention, and feature-reinforced error
correction. Our framework adopts a modular design with good generalization
capability. First, the feature-weighting mechanism assigns input features with
temporal weights. Second, a recurrent encoder-decoder structure with
hierarchical attention is developed as a load predictor. The hierarchical
attention enables a similar day selection, which re-evaluates the importance of
historical information at each time step. Third, we develop an error correction
module that explores the errors and learned feature hidden information to
further improve the model's forecasting performance. Experimental results
demonstrate that our proposed framework outperforms existing methods on two
public datasets and performance metrics, with the feature weighting mechanism
and error correction module being critical to achieving superior performance.
Our framework provides an effective solution to the electric load forecasting
problem, which can be further adapted to many other forecasting tasks.
Related papers
- Localized Gaussians as Self-Attention Weights for Point Clouds Correspondence [92.07601770031236]
We investigate semantically meaningful patterns in the attention heads of an encoder-only Transformer architecture.
We find that fixing the attention weights not only accelerates the training process but also enhances the stability of the optimization.
arXiv Detail & Related papers (2024-09-20T07:41:47Z) - Explanatory Model Monitoring to Understand the Effects of Feature Shifts on Performance [61.06245197347139]
We propose a novel approach to explain the behavior of a black-box model under feature shifts.
We refer to our method that combines concepts from Optimal Transport and Shapley Values as Explanatory Performance Estimation.
arXiv Detail & Related papers (2024-08-24T18:28:19Z) - Spatiotemporal Observer Design for Predictive Learning of
High-Dimensional Data [6.214987339902511]
An observer theory-guided deep learning architecture, called Stemporal, is designed for predictive learning Observer high dimensional data.
This framework could capture thetemporaltemporal dynamics make accurate predictions in both one-step and multi-step-ahead scenarios.
arXiv Detail & Related papers (2024-02-23T12:28:31Z) - Enhanced LFTSformer: A Novel Long-Term Financial Time Series Prediction Model Using Advanced Feature Engineering and the DS Encoder Informer Architecture [0.8532753451809455]
This study presents a groundbreaking model for forecasting long-term financial time series, termed the Enhanced LFTSformer.
The model distinguishes itself through several significant innovations.
Systematic experimentation on a range of benchmark stock market datasets demonstrates that the Enhanced LFTSformer outperforms traditional machine learning models.
arXiv Detail & Related papers (2023-10-03T08:37:21Z) - Short-Term Load Forecasting Using A Particle-Swarm Optimized Multi-Head
Attention-Augmented CNN-LSTM Network [0.0]
Short-term load forecasting is of paramount importance in the efficient operation and planning of power systems.
Recent strides in deep learning have shown promise in addressing this challenge.
I propose a novel solution that surmounts these obstacles.
arXiv Detail & Related papers (2023-09-07T13:06:52Z) - EvCenterNet: Uncertainty Estimation for Object Detection using
Evidential Learning [26.535329379980094]
EvCenterNet is a novel uncertainty-aware 2D object detection framework.
We employ evidential learning to estimate both classification and regression uncertainties.
We train our model on the KITTI dataset and evaluate it on challenging out-of-distribution datasets.
arXiv Detail & Related papers (2023-03-06T11:07:11Z) - Improving Adaptive Conformal Prediction Using Self-Supervised Learning [72.2614468437919]
We train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores.
We empirically demonstrate the benefit of the additional information using both synthetic and real data on the efficiency (width), deficit, and excess of conformal prediction intervals.
arXiv Detail & Related papers (2023-02-23T18:57:14Z) - Robust Graph Representation Learning via Predictive Coding [46.22695915912123]
Predictive coding is a message-passing framework initially developed to model information processing in the brain.
In this work, we build models that rely on the message-passing rule of predictive coding.
We show that the proposed models are comparable to standard ones in terms of performance in both inductive and transductive tasks.
arXiv Detail & Related papers (2022-12-09T03:58:22Z) - Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate
Model Predictive Trajectory Tracking [76.27433308688592]
Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation.
We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience.
Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
arXiv Detail & Related papers (2022-06-07T13:51:35Z) - Leveraging the structure of dynamical systems for data-driven modeling [111.45324708884813]
We consider the impact of the training set and its structure on the quality of the long-term prediction.
We show how an informed design of the training set, based on invariants of the system and the structure of the underlying attractor, significantly improves the resulting models.
arXiv Detail & Related papers (2021-12-15T20:09:20Z) - Forethought and Hindsight in Credit Assignment [62.05690959741223]
We work to understand the gains and peculiarities of planning employed as forethought via forward models or as hindsight operating with backward models.
We investigate the best use of models in planning, primarily focusing on the selection of states in which predictions should be (re)-evaluated.
arXiv Detail & Related papers (2020-10-26T16:00:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.