TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis

• URL: http://arxiv.org/abs/2402.16412v1
• Date: Mon, 26 Feb 2024 09:11:12 GMT
• Title: TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis
• Authors: Sabera Talukder and Yisong Yue and Georgia Gkioxari
• Abstract summary: We propose a simple tokenizer architecture that embeds time series data from varying domains using a discrete vectorized representation learned in a self-supervised manner. We study the efficacy of TOTEM with an extensive evaluation on 17 real world time series datasets across 3 tasks.
• Score: 32.854449155765344
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The field of general time series analysis has recently begun to explore unified modeling, where a common architectural backbone can be retrained on a specific task for a specific dataset. In this work, we approach unification from a complementary vantage point: unification across tasks and domains. To this end, we explore the impact of discrete, learnt, time series data representations that enable generalist, cross-domain training. Our method, TOTEM, or TOkenized Time Series EMbeddings, proposes a simple tokenizer architecture that embeds time series data from varying domains using a discrete vectorized representation learned in a self-supervised manner. TOTEM works across multiple tasks and domains with minimal to no tuning. We study the efficacy of TOTEM with an extensive evaluation on 17 real world time series datasets across 3 tasks. We evaluate both the specialist (i.e., training a model on each domain) and generalist (i.e., training a single model on many domains) settings, and show that TOTEM matches or outperforms previous best methods on several popular benchmarks. The code can be found at: https://github.com/SaberaTalukder/TOTEM.

Related papers

• Deep Time Series Models: A Comprehensive Survey and Benchmark [74.28364194333447]
  Time series data is of great significance in real-world scenarios. Recent years have witnessed remarkable breakthroughs in the time series community. We release Time Series Library (TSLib) as a fair benchmark of deep time series models for diverse analysis tasks.
  arXiv  Detail & Related papers  (2024-07-18T08:31:55Z)
• Capturing Temporal Components for Time Series Classification [5.70772577110828]
  This work introduces a textitcompositional representation learning approach trained on statistically coherent components extracted from sequential data. Based on a multi-scale change space, an unsupervised approach is proposed to segment the sequential data into chunks with similar statistical properties. A sequence-based encoder model is trained in a multi-task setting to learn compositional representations from these temporal components for time series classification.
  arXiv  Detail & Related papers  (2024-06-20T16:15:21Z)
• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• NuwaTS: a Foundation Model Mending Every Incomplete Time Series [24.768755438620666]
  We present textbfNuwaTS, a novel framework that repurposes Pre-trained Language Models for general time series imputation. NuwaTS can be applied to impute missing data across any domain. We show that NuwaTS generalizes to other time series tasks, such as forecasting.
  arXiv  Detail & Related papers  (2024-05-24T07:59:02Z)
• UniCL: A Universal Contrastive Learning Framework for Large Time Series Models [18.005358506435847]
  Time-series analysis plays a pivotal role across a range of critical applications, from finance to healthcare. Traditional supervised learning methods first annotate extensive labels for time-series data in each task. This paper introduces UniCL, a universal and scalable contrastive learning framework designed for pretraining time-series foundation models.
  arXiv  Detail & Related papers  (2024-05-17T07:47:11Z)
• Large Pre-trained time series models for cross-domain Time series analysis tasks [20.228846068418765]
  We propose a novel method of textitadaptive segmentation that automatically identifies optimal dataset-specific segmentation strategy during pre-training. This enables LPTM to perform similar to or better than domain-specific state-of-art model when fine-tuned to different downstream time-series analysis tasks and under zero-shot settings.
  arXiv  Detail & Related papers  (2023-11-19T20:16:16Z)
• Temporal Treasure Hunt: Content-based Time Series Retrieval System for Discovering Insights [34.1973242428317]
  Time series data is ubiquitous across various domains such as finance, healthcare, and manufacturing. The ability to perform Content-based Time Series Retrieval (CTSR) is crucial for identifying unknown time series examples. We introduce a CTSR benchmark dataset that comprises time series data from a variety of domains.
  arXiv  Detail & Related papers  (2023-11-05T04:12:13Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• Novel Features for Time Series Analysis: A Complex Networks Approach [62.997667081978825]
  Time series data are ubiquitous in several domains as climate, economics and health care. Recent conceptual approach relies on time series mapping to complex networks. Network analysis can be used to characterize different types of time series.
  arXiv  Detail & Related papers  (2021-10-11T13:46:28Z)
• Time Series Analysis via Network Science: Concepts and Algorithms [62.997667081978825]
  This review provides a comprehensive overview of existing mapping methods for transforming time series into networks. We describe the main conceptual approaches, provide authoritative references and give insight into their advantages and limitations in a unified notation and language. Although still very recent, this research area has much potential and with this survey we intend to pave the way for future research on the topic.
  arXiv  Detail & Related papers  (2021-10-11T13:33:18Z)
• Simple multi-dataset detection [83.9604523643406]
  We present a simple method for training a unified detector on multiple large-scale datasets. We show how to automatically integrate dataset-specific outputs into a common semantic taxonomy. Our approach does not require manual taxonomy reconciliation.
  arXiv  Detail & Related papers  (2021-02-25T18:55:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.