TimeXer: Empowering Transformers for Time Series Forecasting with
Exogenous Variables
- URL: http://arxiv.org/abs/2402.19072v1
- Date: Thu, 29 Feb 2024 11:54:35 GMT
- Title: TimeXer: Empowering Transformers for Time Series Forecasting with
Exogenous Variables
- Authors: Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Yong Liu, Yunzhong Qiu, Haoran
Zhang, Jianmin Wang, Mingsheng Long
- Abstract summary: We propose a novel framework, TimeXer, to utilize external information to enhance the forecasting of endogenous variables.
TimeXer significantly improves time series forecasting with endogenous variables and achieves consistent state-of-the-art performance in twelve real-world forecasting benchmarks.
- Score: 82.07393844821522
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Recent studies have demonstrated remarkable performance in time series
forecasting. However, due to the partially-observed nature of real-world
applications, solely focusing on the target of interest, so-called endogenous
variables, is usually insufficient to guarantee accurate forecasting. Notably,
a system is often recorded into multiple variables, where the exogenous series
can provide valuable external information for endogenous variables. Thus,
unlike prior well-established multivariate or univariate forecasting that
either treats all the variables equally or overlooks exogenous information,
this paper focuses on a practical setting, which is time series forecasting
with exogenous variables. We propose a novel framework, TimeXer, to utilize
external information to enhance the forecasting of endogenous variables. With a
deftly designed embedding layer, TimeXer empowers the canonical Transformer
architecture with the ability to reconcile endogenous and exogenous
information, where patch-wise self-attention and variate-wise cross-attention
are employed. Moreover, a global endogenous variate token is adopted to
effectively bridge the exogenous series into endogenous temporal patches.
Experimentally, TimeXer significantly improves time series forecasting with
exogenous variables and achieves consistent state-of-the-art performance in
twelve real-world forecasting benchmarks.
Related papers
- Leveraging 2D Information for Long-term Time Series Forecasting with Vanilla Transformers [55.475142494272724]
Time series prediction is crucial for understanding and forecasting complex dynamics in various domains.
We introduce GridTST, a model that combines the benefits of two approaches using innovative multi-directional attentions.
The model consistently delivers state-of-the-art performance across various real-world datasets.
arXiv Detail & Related papers (2024-05-22T16:41:21Z) - Unified Training of Universal Time Series Forecasting Transformers [104.56318980466742]
We present a Masked-based Universal Time Series Forecasting Transformer (Moirai)
Moirai is trained on our newly introduced Large-scale Open Time Series Archive (LOTSA) featuring over 27B observations across nine domains.
Moirai achieves competitive or superior performance as a zero-shot forecaster when compared to full-shot models.
arXiv Detail & Related papers (2024-02-04T20:00:45Z) - Performative Time-Series Forecasting [71.18553214204978]
We formalize performative time-series forecasting (PeTS) from a machine-learning perspective.
We propose a novel approach, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts.
We conduct comprehensive experiments using multiple time-series models on COVID-19 and traffic forecasting tasks.
arXiv Detail & Related papers (2023-10-09T18:34:29Z) - W-Transformers : A Wavelet-based Transformer Framework for Univariate
Time Series Forecasting [7.075125892721573]
We build a transformer model for non-stationary time series using wavelet-based transformer encoder architecture.
We evaluate our framework on several publicly available benchmark time series datasets from various domains.
arXiv Detail & Related papers (2022-09-08T17:39:38Z) - DRAformer: Differentially Reconstructed Attention Transformer for
Time-Series Forecasting [7.805077630467324]
Time-series forecasting plays an important role in many real-world scenarios, such as equipment life cycle forecasting, weather forecasting, and traffic flow forecasting.
It can be observed from recent research that a variety of transformer-based models have shown remarkable results in time-series forecasting.
However, there are still some issues that limit the ability of transformer-based models on time-series forecasting tasks.
arXiv Detail & Related papers (2022-06-11T10:34:29Z) - TACTiS: Transformer-Attentional Copulas for Time Series [76.71406465526454]
estimation of time-varying quantities is a fundamental component of decision making in fields such as healthcare and finance.
We propose a versatile method that estimates joint distributions using an attention-based decoder.
We show that our model produces state-of-the-art predictions on several real-world datasets.
arXiv Detail & Related papers (2022-02-07T21:37:29Z) - ETSformer: Exponential Smoothing Transformers for Time-series
Forecasting [35.76867542099019]
We propose ETSFormer, a novel time-series Transformer architecture, which exploits the principle of exponential smoothing in improving Transformers for time-series forecasting.
In particular, inspired by the classical exponential smoothing methods in time-series forecasting, we propose the novel exponential smoothing attention (ESA) and frequency attention (FA) to replace the self-attention mechanism in vanilla Transformers, thus improving both accuracy and efficiency.
arXiv Detail & Related papers (2022-02-03T02:50:44Z) - Instance-wise Graph-based Framework for Multivariate Time Series
Forecasting [69.38716332931986]
We propose a simple yet efficient instance-wise graph-based framework to utilize the inter-dependencies of different variables at different time stamps.
The key idea of our framework is aggregating information from the historical time series of different variables to the current time series that we need to forecast.
arXiv Detail & Related papers (2021-09-14T07:38:35Z) - Deep Transformer Models for Time Series Forecasting: The Influenza
Prevalence Case [2.997238772148965]
Time series data are prevalent in many scientific and engineering disciplines.
We present a new approach to time series forecasting using Transformer-based machine learning models.
We show that the forecasting results produced by our approach are favorably comparable to the state-of-the-art.
arXiv Detail & Related papers (2020-01-23T00:22:22Z) - Temporal Tensor Transformation Network for Multivariate Time Series
Prediction [1.2354076490479515]
We present a novel deep learning architecture, known as Temporal Transformation Network, which transforms the original time series into a higher order.
This yields a new representation of the original multivariate time series, which enables the convolution kernel to extract complex and non-linear features as well as variable interactional signals from a relatively large temporal region.
Experimental results show that Temporal Transformation Network outperforms several state-of-the-art methods on window-based predictions across various tasks.
arXiv Detail & Related papers (2020-01-04T07:28:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.