General Time-series Model for Universal Knowledge Representation of Multivariate Time-Series data

• URL: http://arxiv.org/abs/2502.03264v1
• Date: Wed, 05 Feb 2025 15:20:04 GMT
• Title: General Time-series Model for Universal Knowledge Representation of Multivariate Time-Series data
• Authors: Cheng He, Xu Huang, Gangwei Jiang, Zhaoyi Li, Defu Lian, Hong Xie, Enhong Chen, Xijie Liang, Zengrong Zheng,
• Abstract summary: We show that time series with different time granularities (or corresponding frequency resolutions) exhibit distinct joint distributions in the frequency domain. A novel Fourier knowledge attention mechanism is proposed to enable learning time-aware representations from both the temporal and frequency domains. An autoregressive blank infilling pre-training framework is incorporated to time series analysis for the first time, leading to a generative tasks agnostic pre-training strategy.
• Score: 61.163542597764796
• License:
• Abstract: Universal knowledge representation is a central problem for multivariate time series(MTS) foundation models and yet remains open. This paper investigates this problem from the first principle and it makes four folds of contributions. First, a new empirical finding is revealed: time series with different time granularities (or corresponding frequency resolutions) exhibit distinct joint distributions in the frequency domain. This implies a crucial aspect of learning universal knowledge, one that has been overlooked by previous studies. Second, a novel Fourier knowledge attention mechanism is proposed to enable learning time granularity-aware representations from both the temporal and frequency domains. Third, an autoregressive blank infilling pre-training framework is incorporated to time series analysis for the first time, leading to a generative tasks agnostic pre-training strategy. To this end, we develop the General Time-series Model (GTM), a unified MTS foundation model that addresses the limitation of contemporary time series models, which often require token, pre-training, or model-level customizations for downstream tasks adaption. Fourth, extensive experiments show that GTM outperforms state-of-the-art (SOTA) methods across all generative tasks, including long-term forecasting, anomaly detection, and imputation.

Related papers

• FlexTSF: A Universal Forecasting Model for Time Series with Variable Regularities [17.164913785452367]
  We propose FlexTSF, a universal time series forecasting model that possesses better generalization and supports both regular and irregular time series. Experiments on 12 datasets show that FlexTSF outperforms state-of-the-art forecasting models respectively designed for regular and irregular time series.
  arXiv  Detail & Related papers  (2024-10-30T16:14:09Z)
• TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis [17.09401448377127]
  Time series analysis plays a critical role in numerous applications, supporting tasks such as forecasting, classification, anomaly detection, and imputation. In this work, we present the time series pattern machine (TSPM), a model designed to excel in a broad range of time series tasks through powerful representation and pattern extraction capabilities.
  arXiv  Detail & Related papers  (2024-10-21T14:06:53Z)
• Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts [103.725112190618]
  This paper introduces Moirai-MoE, using a single input/output projection layer while delegating the modeling of diverse time series patterns to the sparse mixture of experts. Extensive experiments on 39 datasets demonstrate the superiority of Moirai-MoE over existing foundation models in both in-distribution and zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-10-14T13:01:11Z)
• Towards Generalisable Time Series Understanding Across Domains [10.350643783811174]
  We introduce a novel pre-training paradigm specifically designed to handle time series heterogeneity. We propose a tokeniser with learnable domain signatures, a dual masking strategy, and a normalised cross-correlation loss. Our code and pre-trained weights are available at https://www.oetu.com/oetu/otis.
  arXiv  Detail & Related papers  (2024-10-09T17:09:30Z)
• Foundation Models for Time Series Analysis: A Tutorial and Survey [70.43311272903334]
  Foundation Models (FMs) have fundamentally reshaped the paradigm of model design for time series analysis. This survey aims to furnish a comprehensive and up-to-date overview of FMs for time series analysis.
  arXiv  Detail & Related papers  (2024-03-21T10:08:37Z)
• Unified Training of Universal Time Series Forecasting Transformers [104.56318980466742]
  We present a Masked-based Universal Time Series Forecasting Transformer (Moirai) Moirai is trained on our newly introduced Large-scale Open Time Series Archive (LOTSA) featuring over 27B observations across nine domains. Moirai achieves competitive or superior performance as a zero-shot forecaster when compared to full-shot models.
  arXiv  Detail & Related papers  (2024-02-04T20:00:45Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• An Unsupervised Short- and Long-Term Mask Representation for Multivariate Time Series Anomaly Detection [2.387411589813086]
  This paper proposes an anomaly detection method based on unsupervised Short- and Long-term Mask Representation learning (SLMR) Experiments show that the performance of our method outperforms other state-of-the-art models on three real-world datasets.
  arXiv  Detail & Related papers  (2022-08-19T09:34:11Z)
• Multi-scale Attention Flow for Probabilistic Time Series Forecasting [68.20798558048678]
  We propose a novel non-autoregressive deep learning model, called Multi-scale Attention Normalizing Flow(MANF) Our model avoids the influence of cumulative error and does not increase the time complexity. Our model achieves state-of-the-art performance on many popular multivariate datasets.
  arXiv  Detail & Related papers  (2022-05-16T07:53:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.