TwinS: Revisiting Non-Stationarity in Multivariate Time Series Forecasting

• URL: http://arxiv.org/abs/2406.03710v2
• Date: Sun, 14 Jul 2024 14:55:16 GMT
• Title: TwinS: Revisiting Non-Stationarity in Multivariate Time Series Forecasting
• Authors: Jiaxi Hu, Qingsong Wen, Sijie Ruan, Li Liu, Yuxuan Liang,
• Abstract summary: We propose a Transformer-based TwinS model to address the non-stationary periodic distributions. The Wavelet Convolution models nested periods by scaling the convolution kernel size like wavelet. The Period-Aware Attention guides attention by generating period relevance scores through a convolutional sub-network. The Channel-Temporal Mixed captures the overall relationships between time series through channel-time mixing learning.
• Score: 29.208648615140575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, multivariate time series forecasting tasks have garnered increasing attention due to their significant practical applications, leading to the emergence of various deep forecasting models. However, real-world time series exhibit pronounced non-stationary distribution characteristics. These characteristics are not solely limited to time-varying statistical properties highlighted by non-stationary Transformer but also encompass three key aspects: nested periodicity, absence of periodic distributions, and hysteresis among time variables. In this paper, we begin by validating this theory through wavelet analysis and propose the Transformer-based TwinS model, which consists of three modules to address the non-stationary periodic distributions: Wavelet Convolution, Period-Aware Attention, and Channel-Temporal Mixed MLP. Specifically, The Wavelet Convolution models nested periods by scaling the convolution kernel size like wavelet transform. The Period-Aware Attention guides attention computation by generating period relevance scores through a convolutional sub-network. The Channel-Temporal Mixed MLP captures the overall relationships between time series through channel-time mixing learning. TwinS achieves SOTA performance compared to mainstream TS models, with a maximum improvement in MSE of 25.8\% over PatchTST.

Related papers

• MFRS: A Multi-Frequency Reference Series Approach to Scalable and Accurate Time-Series Forecasting [51.94256702463408]
  Time series predictability is derived from periodic characteristics at different frequencies. We propose a novel time series forecasting method based on multi-frequency reference series correlation analysis. Experiments on major open and synthetic datasets show state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-11T11:40:14Z)
• Lightweight Channel-wise Dynamic Fusion Model: Non-stationary Time Series Forecasting via Entropy Analysis [25.291749176117662]
  We show that variance can be a valid and interpretable proxy for non-stationarity of time series. We propose a novel lightweight textitChannel-wise textitDynamic textitFusion textitModel (textitCDFM) Comprehensive experiments on seven time series datasets demonstrate the superiority and generalization capabilities of CDFM.
  arXiv  Detail & Related papers  (2025-03-04T13:29:42Z)
• Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts [103.725112190618]
  This paper introduces Moirai-MoE, using a single input/output projection layer while delegating the modeling of diverse time series patterns to the sparse mixture of experts. Extensive experiments on 39 datasets demonstrate the superiority of Moirai-MoE over existing foundation models in both in-distribution and zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-10-14T13:01:11Z)
• Timer-XL: Long-Context Transformers for Unified Time Series Forecasting [67.83502953961505]
  We present Timer-XL, a generative Transformer for unified time series forecasting. Timer-XL achieves state-of-the-art performance across challenging forecasting benchmarks through a unified approach.
  arXiv  Detail & Related papers  (2024-10-07T07:27:39Z)
• Robust Multivariate Time Series Forecasting against Intra- and Inter-Series Transitional Shift [40.734564394464556]
  We present a unified Probabilistic Graphical Model to Jointly capturing intra-/inter-series correlations and modeling the time-variant transitional distribution. We validate the effectiveness and efficiency of JointPGM through extensive experiments on six highly non-stationary MTS datasets.
  arXiv  Detail & Related papers  (2024-07-18T06:16:03Z)
• FAITH: Frequency-domain Attention In Two Horizons for Time Series Forecasting [13.253624747448935]
  Time Series Forecasting plays a crucial role in various fields such as industrial equipment maintenance, meteorology, energy consumption, traffic flow and financial investment. Current deep learning-based predictive models often exhibit a significant deviation between their forecasting outcomes and the ground truth. We propose a novel model Frequency-domain Attention In Two Horizons, which decomposes time series into trend and seasonal components.
  arXiv  Detail & Related papers  (2024-05-22T02:37:02Z)
• Frequency-domain MLPs are More Effective Learners in Time Series Forecasting [67.60443290781988]
  Time series forecasting has played the key role in different industrial domains, including finance, traffic, energy, and healthcare. Most-based forecasting methods suffer from the point-wise mappings and information bottleneck. We propose FreTS, a simple yet effective architecture built upon Frequency-domains for Time Series forecasting.
  arXiv  Detail & Related papers  (2023-11-10T17:05:13Z)
• CARD: Channel Aligned Robust Blend Transformer for Time Series Forecasting [50.23240107430597]
  We design a special Transformer, i.e., Channel Aligned Robust Blend Transformer (CARD for short), that addresses key shortcomings of CI type Transformer in time series forecasting. First, CARD introduces a channel-aligned attention structure that allows it to capture both temporal correlations among signals. Second, in order to efficiently utilize the multi-scale knowledge, we design a token blend module to generate tokens with different resolutions. Third, we introduce a robust loss function for time series forecasting to alleviate the potential overfitting issue.
  arXiv  Detail & Related papers  (2023-05-20T05:16:31Z)
• MTS-Mixers: Multivariate Time Series Forecasting via Factorized Temporal and Channel Mixing [18.058617044421293]
  This paper investigates the contributions and deficiencies of attention mechanisms on the performance of time series forecasting. We propose MTS-Mixers, which use two factorized modules to capture temporal and channel dependencies. Experimental results on several real-world datasets show that MTS-Mixers outperform existing Transformer-based models with higher efficiency.
  arXiv  Detail & Related papers  (2023-02-09T08:52:49Z)
• Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement [51.55157852647306]
  Time series forecasting has been a widely explored task of great importance in many applications. It is common that real-world time series data are recorded in a short time period, which results in a big gap between the deep model and the limited and noisy time series. We propose to address the time series forecasting problem with generative modeling and propose a bidirectional variational auto-encoder equipped with diffusion, denoise, and disentanglement.
  arXiv  Detail & Related papers  (2023-01-08T12:20:46Z)
• Features Fusion Framework for Multimodal Irregular Time-series Events [6.497816402045097]
  multimodal irregular time-series events have different sampling frequencies, data compositions, temporal relations and characteristics. In this paper, a features fusion framework for multimodal irregular time-series events is proposed based on the Long Short-Term Memory networks (LSTM) Experiments on MIMIC-III dataset demonstrate that the proposed framework significantly outperforms to the existing methods in terms of AUC (the area under Receiver Operating Characteristic curve) and AP (Average Precision)
  arXiv  Detail & Related papers  (2022-09-05T02:27:12Z)
• Distributional Drift Adaptation with Temporal Conditional Variational Autoencoder for Multivariate Time Series Forecasting [41.206310481507565]
  We propose a novel framework temporal conditional variational autoencoder (TCVAE) to model the dynamic distributional dependencies over time. The TCVAE infers the dependencies as a temporal conditional distribution to leverage latent variables. We show the TCVAE's superior robustness and effectiveness over the state-of-the-art MTS forecasting baselines.
  arXiv  Detail & Related papers  (2022-09-01T10:06:22Z)
• Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting [86.33543833145457]
  We propose Non-stationary Transformers as a generic framework with two interdependent modules: Series Stationarization and De-stationary Attention. Our framework consistently boosts mainstream Transformers by a large margin, which reduces MSE by 49.43% on Transformer, 47.34% on Informer, and 46.89% on Reformer.
  arXiv  Detail & Related papers  (2022-05-28T12:27:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.