Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects

• URL: http://arxiv.org/abs/2306.10125v4
• Date: Mon, 8 Apr 2024 15:38:59 GMT
• Title: Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects
• Authors: Kexin Zhang, Qingsong Wen, Chaoli Zhang, Rongyao Cai, Ming Jin, Yong Liu, James Zhang, Yuxuan Liang, Guansong Pang, Dongjin Song, Shirui Pan,
• Abstract summary: Self-supervised learning (SSL) has recently achieved impressive performance on various time series tasks. This article reviews current state-of-the-art SSL methods for time series data.
• Score: 84.6945070729684
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervised learning (SSL) has recently achieved impressive performance on various time series tasks. The most prominent advantage of SSL is that it reduces the dependence on labeled data. Based on the pre-training and fine-tuning strategy, even a small amount of labeled data can achieve high performance. Compared with many published self-supervised surveys on computer vision and natural language processing, a comprehensive survey for time series SSL is still missing. To fill this gap, we review current state-of-the-art SSL methods for time series data in this article. To this end, we first comprehensively review existing surveys related to SSL and time series, and then provide a new taxonomy of existing time series SSL methods by summarizing them from three perspectives: generative-based, contrastive-based, and adversarial-based. These methods are further divided into ten subcategories with detailed reviews and discussions about their key intuitions, main frameworks, advantages and disadvantages. To facilitate the experiments and validation of time series SSL methods, we also summarize datasets commonly used in time series forecasting, classification, anomaly detection, and clustering tasks. Finally, we present the future directions of SSL for time series analysis.

Related papers

• A Survey of the Self Supervised Learning Mechanisms for Vision Transformers [5.152455218955949]
  The application of self supervised learning (SSL) in vision tasks has gained significant attention. We develop a comprehensive taxonomy of systematically classifying the SSL techniques. We discuss the motivations behind SSL, review popular pre-training tasks, and highlight the challenges and advancements in this field.
  arXiv  Detail & Related papers  (2024-08-30T07:38:28Z)
• Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning [4.137391543972184]
  Semi-supervised learning (SSL) has witnessed remarkable progress, resulting in numerous method variations. In this paper, we present a novel SSL approach named FineSSL that significantly addresses this limitation by adapting pre-trained foundation models. We demonstrate that FineSSL sets a new state of the art for SSL on multiple benchmark datasets, reduces the training cost by over six times, and can seamlessly integrate various fine-tuning and modern SSL algorithms.
  arXiv  Detail & Related papers  (2024-05-20T03:33:12Z)
• TimeCSL: Unsupervised Contrastive Learning of General Shapelets for Explorable Time Series Analysis [33.137110972937855]
  TimeCSL is an end-to-end system that makes full use of the general and interpretable shapelets learned by CSL to achieve explorable time series analysis. We introduce the system components and demonstrate how users interact with TimeCSL to solve different analysis tasks in the unified pipeline.
  arXiv  Detail & Related papers  (2024-04-07T19:39:14Z)
• Self-Supervised Learning for Time Series: Contrastive or Generative? [7.712601563682029]
  Self-supervised learning (SSL) has emerged as a powerful approach to learning representations from large-scale unlabeled data. We will present a comprehensive comparative study between contrastive and generative methods in time series. Our results provide insights into the strengths and weaknesses of each approach and offer practical recommendations for choosing suitable SSL methods.
  arXiv  Detail & Related papers  (2024-03-14T18:58:06Z)
• Empowering Time Series Analysis with Large Language Models: A Survey [24.202539098675953]
  We provide a systematic overview of methods that leverage large language models for time series analysis. Specifically, we first state the challenges and motivations of applying language models in the context of time series. Next, we categorize existing methods into different groups (i.e., direct query, tokenization, prompt design, fine-tune, and model integration) and highlight the key ideas within each group.
  arXiv  Detail & Related papers  (2024-02-05T16:46:35Z)
• Spatiotemporal Self-supervised Learning for Point Clouds in the Wild [65.56679416475943]
  We introduce an SSL strategy that leverages positive pairs in both the spatial and temporal domain. We demonstrate the benefits of our approach via extensive experiments performed by self-supervised training on two large-scale LiDAR datasets.
  arXiv  Detail & Related papers  (2023-03-28T18:06:22Z)
• A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends [82.64268080902742]
  Self-supervised learning (SSL) aims to learn discriminative features from unlabeled data without relying on human-annotated labels. SSL has garnered significant attention recently, leading to the development of numerous related algorithms. This paper presents a review of diverse SSL methods, encompassing algorithmic aspects, application domains, three key trends, and open research questions.
  arXiv  Detail & Related papers  (2023-01-13T14:41:05Z)
• Graph-based Semi-supervised Learning: A Comprehensive Review [51.26862262550445]
  Semi-supervised learning (SSL) has tremendous value in practice due to its ability to utilize both labeled data and unlabelled data. An important class of SSL methods is to naturally represent data as graphs, which corresponds to graph-based semi-supervised learning (GSSL) methods. GSSL methods have demonstrated their advantages in various domains due to their uniqueness of structure, the universality of applications, and their scalability to large scale data.
  arXiv  Detail & Related papers  (2021-02-26T05:11:09Z)
• On Data-Augmentation and Consistency-Based Semi-Supervised Learning [77.57285768500225]
  Recently proposed consistency-based Semi-Supervised Learning (SSL) methods have advanced the state of the art in several SSL tasks. Despite these advances, the understanding of these methods is still relatively limited.
  arXiv  Detail & Related papers  (2021-01-18T10:12:31Z)
• Learning from Very Few Samples: A Survey [80.06120185496403]
  Few sample learning is significant and challenging in the field of machine learning. Few sample learning algorithms typically entail hundreds or thousands of supervised samples to guarantee generalization ability.
  arXiv  Detail & Related papers  (2020-09-06T06:13:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.