TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables

• URL: http://arxiv.org/abs/2402.19072v4
• Date: Mon, 11 Nov 2024 03:18:32 GMT
• Title: TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables
• Authors: Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Guo Qin, Haoran Zhang, Yong Liu, Yunzhong Qiu, Jianmin Wang, Mingsheng Long,
• Abstract summary: TimeXer ingests external information to enhance the forecasting of endogenous variables. TimeXer achieves consistent state-of-the-art performance on twelve real-world forecasting benchmarks.
• Score: 75.83318701911274
• License:
• Abstract: Deep models have demonstrated remarkable performance in time series forecasting. However, due to the partially-observed nature of real-world applications, solely focusing on the target of interest, so-called endogenous variables, is usually insufficient to guarantee accurate forecasting. Notably, a system is often recorded into multiple variables, where the exogenous variables can provide valuable external information for endogenous variables. Thus, unlike well-established multivariate or univariate forecasting paradigms that either treat all the variables equally or ignore exogenous information, this paper focuses on a more practical setting: time series forecasting with exogenous variables. We propose a novel approach, TimeXer, to ingest external information to enhance the forecasting of endogenous variables. With deftly designed embedding layers, TimeXer empowers the canonical Transformer with the ability to reconcile endogenous and exogenous information, where patch-wise self-attention and variate-wise cross-attention are used simultaneously. Moreover, global endogenous tokens are learned to effectively bridge the causal information underlying exogenous series into endogenous temporal patches. Experimentally, TimeXer achieves consistent state-of-the-art performance on twelve real-world forecasting benchmarks and exhibits notable generality and scalability. Code is available at this repository: https://github.com/thuml/TimeXer.

Related papers

• Powerformer: A Transformer with Weighted Causal Attention for Time-series Forecasting [50.298817606660826]
  We introduce Powerformer, a novel Transformer variant that replaces noncausal attention weights with causal weights that are reweighted according to a smooth heavy-tailed decay. Our empirical results demonstrate that Powerformer achieves state-of-the-art accuracy on public time-series benchmarks. Our analyses show that the model's locality bias is amplified during training, demonstrating an interplay between time-series data and power-law-based attention.
  arXiv  Detail & Related papers  (2025-02-10T04:42:11Z)
• EDformer: Embedded Decomposition Transformer for Interpretable Multivariate Time Series Predictions [4.075971633195745]
  This paper introduces an embedded transformer, 'EDformer', for time series forecasting tasks. Without altering the fundamental elements, we reuse the Transformer architecture and consider the capable functions of its constituent parts. The model obtains state-of-the-art predicting results in terms of accuracy and efficiency on complex real-world time series datasets.
  arXiv  Detail & Related papers  (2024-12-16T11:13:57Z)
• ExoTST: Exogenous-Aware Temporal Sequence Transformer for Time Series Prediction [11.511830352094353]
  We propose ExoTST, a transformer-based framework for time series prediction. To integrate past and current variables, ExoTST introduces a novel cross-temporal modality fusion module. Experiments on real-world carbon flux datasets and time series benchmarks demonstrate ExoTST's superior performance.
  arXiv  Detail & Related papers  (2024-10-16T03:04:37Z)
• Timer-XL: Long-Context Transformers for Unified Time Series Forecasting [67.83502953961505]
  We present Timer-XL, a generative Transformer for unified time series forecasting. Timer-XL achieves state-of-the-art performance across challenging forecasting benchmarks through a unified approach.
  arXiv  Detail & Related papers  (2024-10-07T07:27:39Z)
• DLFormer: Enhancing Explainability in Multivariate Time Series Forecasting using Distributed Lag Embedding [4.995397953581609]
  This study introduces DLFormer, an attention-based architecture integrated with distributed lag embedding. It showcases superior performance improvements compared to existing attention-based high-performance models.
  arXiv  Detail & Related papers  (2024-08-29T20:39:54Z)
• Leveraging 2D Information for Long-term Time Series Forecasting with Vanilla Transformers [55.475142494272724]
  Time series prediction is crucial for understanding and forecasting complex dynamics in various domains. We introduce GridTST, a model that combines the benefits of two approaches using innovative multi-directional attentions. The model consistently delivers state-of-the-art performance across various real-world datasets.
  arXiv  Detail & Related papers  (2024-05-22T16:41:21Z)
• Probabilistic Learning of Multivariate Time Series with Temporal Irregularity [21.361823581838355]
  Real-world time series often suffer from temporal irregularities, including nonuniform intervals and misaligned variables. We propose an end-to-end framework that models temporal irregularities while capturing the joint distribution of variables at arbitrary continuous-time points.
  arXiv  Detail & Related papers  (2023-06-15T14:08:48Z)
• TACTiS: Transformer-Attentional Copulas for Time Series [76.71406465526454]
  estimation of time-varying quantities is a fundamental component of decision making in fields such as healthcare and finance. We propose a versatile method that estimates joint distributions using an attention-based decoder. We show that our model produces state-of-the-art predictions on several real-world datasets.
  arXiv  Detail & Related papers  (2022-02-07T21:37:29Z)
• Instance-wise Graph-based Framework for Multivariate Time Series Forecasting [69.38716332931986]
  We propose a simple yet efficient instance-wise graph-based framework to utilize the inter-dependencies of different variables at different time stamps. The key idea of our framework is aggregating information from the historical time series of different variables to the current time series that we need to forecast.
  arXiv  Detail & Related papers  (2021-09-14T07:38:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.