Related papers: Temporal Saliency Detection Towards Explainable Transformer-based Timeseries Forecasting

Temporal Saliency Detection Towards Explainable Transformer-based Timeseries Forecasting

URL: http://arxiv.org/abs/2212.07771v3
Date: Fri, 15 Sep 2023 08:31:09 GMT
Title: Temporal Saliency Detection Towards Explainable Transformer-based Timeseries Forecasting
Authors: Nghia Duong-Trung, Duc-Manh Nguyen, Danh Le-Phuoc
Abstract summary: This paper introduces Temporal Saliency Detection (TSD), an effective approach that builds upon the attention mechanism and applies it to multi-horizon time series prediction. The TSD approach facilitates the multiresolution analysis of saliency patterns by condensing multi-heads, thereby progressively enhancing the forecasting of complex time series data.
Score: 3.046315755726937
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Abstract: Despite the notable advancements in numerous Transformer-based models, the task of long multi-horizon time series forecasting remains a persistent challenge, especially towards explainability. Focusing on commonly used saliency maps in explaining DNN in general, our quest is to build attention-based architecture that can automatically encode saliency-related temporal patterns by establishing connections with appropriate attention heads. Hence, this paper introduces Temporal Saliency Detection (TSD), an effective approach that builds upon the attention mechanism and applies it to multi-horizon time series prediction. While our proposed architecture adheres to the general encoder-decoder structure, it undergoes a significant renovation in the encoder component, wherein we incorporate a series of information contracting and expanding blocks inspired by the U-Net style architecture. The TSD approach facilitates the multiresolution analysis of saliency patterns by condensing multi-heads, thereby progressively enhancing the forecasting of complex time series data. Empirical evaluations illustrate the superiority of our proposed approach compared to other models across multiple standard benchmark datasets in diverse far-horizon forecasting settings. The initial TSD achieves substantial relative improvements of 31% and 46% over several models in the context of multivariate and univariate prediction. We believe the comprehensive investigations presented in this study will offer valuable insights and benefits to future research endeavors.