A Shapelet-based Framework for Unsupervised Multivariate Time Series Representation Learning

• URL: http://arxiv.org/abs/2305.18888v4
• Date: Sat, 17 Aug 2024 17:05:29 GMT
• Title: A Shapelet-based Framework for Unsupervised Multivariate Time Series Representation Learning
• Authors: Zhiyu Liang, Jianfeng Zhang, Chen Liang, Hongzhi Wang, Zheng Liang, Lujia Pan,
• Abstract summary: We propose a novel URL framework for multivariate time series by learning time-series-specific shapelet-based representation. To the best of our knowledge, this is the first work that explores the shapelet-based embedding in the unsupervised general-purpose representation learning. A unified shapelet-based encoder and a novel learning objective with multi-grained contrasting and multi-scale alignment are particularly designed to achieve our goal.
• Score: 29.511632089649552
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent studies have shown great promise in unsupervised representation learning (URL) for multivariate time series, because URL has the capability in learning generalizable representation for many downstream tasks without using inaccessible labels. However, existing approaches usually adopt the models originally designed for other domains (e.g., computer vision) to encode the time series data and {rely on strong assumptions to design learning objectives, which limits their ability to perform well}. To deal with these problems, we propose a novel URL framework for multivariate time series by learning time-series-specific shapelet-based representation through a popular contrasting learning paradigm. To the best of our knowledge, this is the first work that explores the shapelet-based embedding in the unsupervised general-purpose representation learning. A unified shapelet-based encoder and a novel learning objective with multi-grained contrasting and multi-scale alignment are particularly designed to achieve our goal, and a data augmentation library is employed to improve the generalization. We conduct extensive experiments using tens of real-world datasets to assess the representation quality on many downstream tasks, including classification, clustering, and anomaly detection. The results demonstrate the superiority of our method against not only URL competitors, but also techniques specially designed for downstream tasks. Our code has been made publicly available at https://github.com/real2fish/CSL.

Related papers

• UniCL: A Universal Contrastive Learning Framework for Large Time Series Models [18.005358506435847]
  Time-series analysis plays a pivotal role across a range of critical applications, from finance to healthcare. Traditional supervised learning methods first annotate extensive labels for time-series data in each task. This paper introduces UniCL, a universal and scalable contrastive learning framework designed for pretraining time-series foundation models.
  arXiv  Detail & Related papers  (2024-05-17T07:47:11Z)
• TimeCSL: Unsupervised Contrastive Learning of General Shapelets for Explorable Time Series Analysis [33.137110972937855]
  TimeCSL is an end-to-end system that makes full use of the general and interpretable shapelets learned by CSL to achieve explorable time series analysis. We introduce the system components and demonstrate how users interact with TimeCSL to solve different analysis tasks in the unified pipeline.
  arXiv  Detail & Related papers  (2024-04-07T19:39:14Z)
• Parametric Augmentation for Time Series Contrastive Learning [33.47157775532995]
  We create positive examples that assist the model in learning robust and discriminative representations. Usually, preset human intuition directs the selection of relevant data augmentations. We propose a contrastive learning framework with parametric augmentation, AutoTCL, which can be adaptively employed to support time series representation learning.
  arXiv  Detail & Related papers  (2024-02-16T03:51:14Z)
• Unsupervised Multi-modal Feature Alignment for Time Series Representation Learning [20.655943795843037]
  We introduce an innovative approach that focuses on aligning and binding time series representations encoded from different modalities. In contrast to conventional methods that fuse features from multiple modalities, our proposed approach simplifies the neural architecture by retaining a single time series encoder. Our approach outperforms existing state-of-the-art URL methods across diverse downstream tasks.
  arXiv  Detail & Related papers  (2023-12-09T22:31:20Z)
• Learning from One Continuous Video Stream [70.30084026960819]
  We introduce a framework for online learning from a single continuous video stream. This poses great challenges given the high correlation between consecutive video frames. We employ pixel-to-pixel modelling as a practical and flexible way to switch between pre-training and single-stream evaluation.
  arXiv  Detail & Related papers  (2023-12-01T14:03:30Z)
• SpawnNet: Learning Generalizable Visuomotor Skills from Pre-trained Networks [52.766795949716986]
  We present a study of the generalization capabilities of the pre-trained visual representations at the categorical level. We propose SpawnNet, a novel two-stream architecture that learns to fuse pre-trained multi-layer representations into a separate network to learn a robust policy.
  arXiv  Detail & Related papers  (2023-07-07T13:01:29Z)
• Ti-MAE: Self-Supervised Masked Time Series Autoencoders [16.98069693152999]
  We propose a novel framework named Ti-MAE, in which the input time series are assumed to follow an integrate distribution. Ti-MAE randomly masks out embedded time series data and learns an autoencoder to reconstruct them at the point-level. Experiments on several public real-world datasets demonstrate that our framework of masked autoencoding could learn strong representations directly from the raw data.
  arXiv  Detail & Related papers  (2023-01-21T03:20:23Z)
• Style-Hallucinated Dual Consistency Learning: A Unified Framework for Visual Domain Generalization [113.03189252044773]
  We propose a unified framework, Style-HAllucinated Dual consistEncy learning (SHADE), to handle domain shift in various visual tasks. Our versatile SHADE can significantly enhance the generalization in various visual recognition tasks, including image classification, semantic segmentation and object detection.
  arXiv  Detail & Related papers  (2022-12-18T11:42:51Z)
• NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research [96.53307645791179]
  We introduce the Never-Ending VIsual-classification Stream (NEVIS'22), a benchmark consisting of a stream of over 100 visual classification tasks. Despite being limited to classification, the resulting stream has a rich diversity of tasks from OCR, to texture analysis, scene recognition, and so forth. Overall, NEVIS'22 poses an unprecedented challenge for current sequential learning approaches due to the scale and diversity of tasks.
  arXiv  Detail & Related papers  (2022-11-15T18:57:46Z)
• X-Learner: Learning Cross Sources and Tasks for Universal Visual Representation [71.51719469058666]
  We propose a representation learning framework called X-Learner. X-Learner learns the universal feature of multiple vision tasks supervised by various sources. X-Learner achieves strong performance on different tasks without extra annotations, modalities and computational costs.
  arXiv  Detail & Related papers  (2022-03-16T17:23:26Z)
• Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations [78.12377360145078]
  Contrastive self-supervised learning has outperformed supervised pretraining on many downstream tasks like segmentation and object detection. In this paper, we first study how biases in the dataset affect existing methods. We show that current contrastive approaches work surprisingly well across: (i) object- versus scene-centric, (ii) uniform versus long-tailed and (iii) general versus domain-specific datasets.
  arXiv  Detail & Related papers  (2021-06-10T17:59:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.