TimeSQL: Improving Multivariate Time Series Forecasting with Multi-Scale Patching and Smooth Quadratic Loss

• URL: http://arxiv.org/abs/2311.11285v1
• Date: Sun, 19 Nov 2023 10:05:50 GMT
• Title: TimeSQL: Improving Multivariate Time Series Forecasting with Multi-Scale Patching and Smooth Quadratic Loss
• Authors: Site Mo, Haoxin Wang, Bixiong Li, Songhai Fan, Yuankai Wu, Xianggen Liu
• Abstract summary: Time series is a sequence of real-valued random variables collected at even intervals of time. Time achieves new state-the-art performance on the eight real-world benchmark datasets.
• Score: 9.71229156211078
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Time series is a special type of sequence data, a sequence of real-valued random variables collected at even intervals of time. The real-world multivariate time series comes with noises and contains complicated local and global temporal dynamics, making it difficult to forecast the future time series given the historical observations. This work proposes a simple and effective framework, coined as TimeSQL, which leverages multi-scale patching and smooth quadratic loss (SQL) to tackle the above challenges. The multi-scale patching transforms the time series into two-dimensional patches with different length scales, facilitating the perception of both locality and long-term correlations in time series. SQL is derived from the rational quadratic kernel and can dynamically adjust the gradients to avoid overfitting to the noises and outliers. Theoretical analysis demonstrates that, under mild conditions, the effect of the noises on the model with SQL is always smaller than that with MSE. Based on the two modules, TimeSQL achieves new state-of-the-art performance on the eight real-world benchmark datasets. Further ablation studies indicate that the key modules in TimeSQL could also enhance the results of other models for multivariate time series forecasting, standing as plug-and-play techniques.

Related papers

• Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts [103.725112190618]
  This paper introduces Moirai-MoE, using a single input/output projection layer while delegating the modeling of diverse time series patterns to the sparse mixture of experts. Extensive experiments on 39 datasets demonstrate the superiority of Moirai-MoE over existing foundation models in both in-distribution and zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-10-14T13:01:11Z)
• Timer-XL: Long-Context Transformers for Unified Time Series Forecasting [67.83502953961505]
  We present Timer-XL, a generative Transformer for unified time series forecasting. Timer-XL achieves state-of-the-art performance across challenging forecasting benchmarks through a unified approach.
  arXiv  Detail & Related papers  (2024-10-07T07:27:39Z)
• Leveraging 2D Information for Long-term Time Series Forecasting with Vanilla Transformers [55.475142494272724]
  Time series prediction is crucial for understanding and forecasting complex dynamics in various domains. We introduce GridTST, a model that combines the benefits of two approaches using innovative multi-directional attentions. The model consistently delivers state-of-the-art performance across various real-world datasets.
  arXiv  Detail & Related papers  (2024-05-22T16:41:21Z)
• TPRNN: A Top-Down Pyramidal Recurrent Neural Network for Time Series Forecasting [7.08506873242564]
  Time series have multi-scale characteristics, i.e., different temporal patterns at different scales. We propose TPRNN, a Top-down Pyramidal Recurrent Neural Network for time series forecasting. TPRNN has achieved the state-of-the-art performance with an average improvement of 8.13% in MSE compared to the best baseline.
  arXiv  Detail & Related papers  (2023-12-11T12:21:45Z)
• TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis [80.56913334060404]
  Time series analysis is of immense importance in applications, such as weather forecasting, anomaly detection, and action recognition. Previous methods attempt to accomplish this directly from the 1D time series. We ravel out the complex temporal variations into the multiple intraperiod- and interperiod-variations.
  arXiv  Detail & Related papers  (2022-10-05T12:19:51Z)
• Expressing Multivariate Time Series as Graphs with Time Series Attention Transformer [14.172091921813065]
  We propose the Time Series Attention Transformer (TSAT) for multivariate time series representation learning. Using TSAT, we represent both temporal information and inter-dependencies of time series in terms of edge-enhanced dynamic graphs. We show that TSAT clearly outerperforms six state-of-the-art baseline methods in various forecasting horizons.
  arXiv  Detail & Related papers  (2022-08-19T12:25:56Z)
• Multi-scale Attention Flow for Probabilistic Time Series Forecasting [68.20798558048678]
  We propose a novel non-autoregressive deep learning model, called Multi-scale Attention Normalizing Flow(MANF) Our model avoids the influence of cumulative error and does not increase the time complexity. Our model achieves state-of-the-art performance on many popular multivariate datasets.
  arXiv  Detail & Related papers  (2022-05-16T07:53:42Z)
• Instance-wise Graph-based Framework for Multivariate Time Series Forecasting [69.38716332931986]
  We propose a simple yet efficient instance-wise graph-based framework to utilize the inter-dependencies of different variables at different time stamps. The key idea of our framework is aggregating information from the historical time series of different variables to the current time series that we need to forecast.
  arXiv  Detail & Related papers  (2021-09-14T07:38:35Z)
• A Deep Structural Model for Analyzing Correlated Multivariate Time Series [11.009809732645888]
  We present a deep learning structural time series model which can handle correlated multivariate time series input. The model explicitly learns/extracts the trend, seasonality, and event components. We compare our model with several state-of-the-art methods through a comprehensive set of experiments on a variety of time series data sets.
  arXiv  Detail & Related papers  (2020-01-02T18:48:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.