Foundation Models for Time Series Analysis: A Tutorial and Survey

• URL: http://arxiv.org/abs/2403.14735v3
• Date: Tue, 18 Jun 2024 08:10:07 GMT
• Title: Foundation Models for Time Series Analysis: A Tutorial and Survey
• Authors: Yuxuan Liang, Haomin Wen, Yuqi Nie, Yushan Jiang, Ming Jin, Dongjin Song, Shirui Pan, Qingsong Wen,
• Abstract summary: Foundation Models (FMs) have fundamentally reshaped the paradigm of model design for time series analysis. This survey aims to furnish a comprehensive and up-to-date overview of FMs for time series analysis.
• Score: 70.43311272903334
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Time series analysis stands as a focal point within the data mining community, serving as a cornerstone for extracting valuable insights crucial to a myriad of real-world applications. Recent advances in Foundation Models (FMs) have fundamentally reshaped the paradigm of model design for time series analysis, boosting various downstream tasks in practice. These innovative approaches often leverage pre-trained or fine-tuned FMs to harness generalized knowledge tailored for time series analysis. This survey aims to furnish a comprehensive and up-to-date overview of FMs for time series analysis. While prior surveys have predominantly focused on either application or pipeline aspects of FMs in time series analysis, they have often lacked an in-depth understanding of the underlying mechanisms that elucidate why and how FMs benefit time series analysis. To address this gap, our survey adopts a methodology-centric classification, delineating various pivotal elements of time-series FMs, including model architectures, pre-training techniques, adaptation methods, and data modalities. Overall, this survey serves to consolidate the latest advancements in FMs pertinent to time series analysis, accentuating their theoretical underpinnings, recent strides in development, and avenues for future exploration.

Related papers

• General Time-series Model for Universal Knowledge Representation of Multivariate Time-Series data [61.163542597764796]
  We show that time series with different time granularities (or corresponding frequency resolutions) exhibit distinct joint distributions in the frequency domain. A novel Fourier knowledge attention mechanism is proposed to enable learning time-aware representations from both the temporal and frequency domains. An autoregressive blank infilling pre-training framework is incorporated to time series analysis for the first time, leading to a generative tasks agnostic pre-training strategy.
  arXiv  Detail & Related papers  (2025-02-05T15:20:04Z)
• TimeRAF: Retrieval-Augmented Foundation model for Zero-shot Time Series Forecasting [59.702504386429126]
  TimeRAF is a Retrieval-Augmented Forecasting model that enhance zero-shot time series forecasting through retrieval-augmented techniques. TimeRAF employs an end-to-end learnable retriever to extract valuable information from the knowledge base.
  arXiv  Detail & Related papers  (2024-12-30T09:06:47Z)
• TSI: A Multi-View Representation Learning Approach for Time Series Forecasting [29.05140751690699]
  This study introduces a novel multi-view approach for time series forecasting. It integrates trend and seasonal representations with an Independent Component Analysis (ICA)-based representation. This approach offers a holistic understanding of time series data, going beyond traditional models that often miss nuanced, nonlinear relationships.
  arXiv  Detail & Related papers  (2024-09-30T02:11:57Z)
• A Survey on Diffusion Models for Time Series and Spatio-Temporal Data [92.1255811066468]
  We review the use of diffusion models in time series and S-temporal data, categorizing them by model, task type, data modality, and practical application domain. We categorize diffusion models into unconditioned and conditioned types discuss time series and S-temporal data separately. Our survey covers their application extensively in various fields including healthcare, recommendation, climate, energy, audio, and transportation.
  arXiv  Detail & Related papers  (2024-04-29T17:19:40Z)
• Empowering Time Series Analysis with Large Language Models: A Survey [24.202539098675953]
  We provide a systematic overview of methods that leverage large language models for time series analysis. Specifically, we first state the challenges and motivations of applying language models in the context of time series. Next, we categorize existing methods into different groups (i.e., direct query, tokenization, prompt design, fine-tune, and model integration) and highlight the key ideas within each group.
  arXiv  Detail & Related papers  (2024-02-05T16:46:35Z)
• A Survey on Deep Learning based Time Series Analysis with Frequency Transformation [74.3919960186696]
  Frequency transformation (FT) has been increasingly incorporated into deep learning models to enhance state-of-the-art accuracy and efficiency in time series analysis. Despite the growing attention and the proliferation of research in this emerging field, there is currently a lack of a systematic review and in-depth analysis of deep learning-based time series models with FT. We present a comprehensive review that systematically investigates and summarizes the recent research advancements in deep learning-based time series analysis with FT.
  arXiv  Detail & Related papers  (2023-02-04T14:33:07Z)
• Time Series Analysis via Network Science: Concepts and Algorithms [62.997667081978825]
  This review provides a comprehensive overview of existing mapping methods for transforming time series into networks. We describe the main conceptual approaches, provide authoritative references and give insight into their advantages and limitations in a unified notation and language. Although still very recent, this research area has much potential and with this survey we intend to pave the way for future research on the topic.
  arXiv  Detail & Related papers  (2021-10-11T13:33:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.