LEAVES: Learning Views for Time-Series Biobehavioral Data in Contrastive Learning

• URL: http://arxiv.org/abs/2210.07340v2
• Date: Tue, 12 Aug 2025 23:03:00 GMT
• Title: LEAVES: Learning Views for Time-Series Biobehavioral Data in Contrastive Learning
• Authors: Han Yu, Huiyuan Yang, Akane Sano,
• Abstract summary: We introduce a module for automatic view generation in contrastive learning frameworks applied to time-series biobehavioral data.<n>We assess the efficacy of our method on multiple time-series datasets using two well-known contrastive learning frameworks.
• Score: 17.693664200804157
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Contrastive learning has been utilized as a promising self-supervised learning approach to extract meaningful representations from unlabeled data. The majority of these methods take advantage of data-augmentation techniques to create diverse views from the original input. However, optimizing augmentations and their parameters for generating more effective views in contrastive learning frameworks is often resource-intensive and time-consuming. While several strategies have been proposed for automatically generating new views in computer vision, research in other domains, such as time-series biobehavioral data, remains limited. In this paper, we introduce a simple yet powerful module for automatic view generation in contrastive learning frameworks applied to time-series biobehavioral data, which is essential for modern health care, termed learning views for time-series data (LEAVES). This proposed module employs adversarial training to learn augmentation hyperparameters within contrastive learning frameworks. We assess the efficacy of our method on multiple time-series datasets using two well-known contrastive learning frameworks, namely SimCLR and BYOL. Across four diverse biobehavioral datasets, LEAVES requires only approximately 20 learnable parameters -- dramatically fewer than the about 580k parameters demanded by frameworks like ViewMaker, a previously proposed adversarially trained convolutional module in contrastive learning, while achieving competitive and often superior performance to existing baseline methods. Crucially, these efficiency gains are obtained without extensive manual hyperparameter tuning, which makes LEAVES particularly suitable for large-scale or real-time healthcare applications that demand both accuracy and practicality.

Related papers

• FreRA: A Frequency-Refined Augmentation for Contrastive Learning on Time Series Classification [56.925103708982164]
  We present a novel perspective from the frequency domain and identify three advantages for downstream classification: global, independent, and compact.<n>We propose the lightweight yet effective Frequency Refined Augmentation (FreRA) tailored for time series contrastive learning on classification tasks.<n>FreRA consistently outperforms ten leading baselines on time series classification, anomaly detection, and transfer learning tasks.
  arXiv  Detail & Related papers  (2025-05-29T07:18:28Z)
• CSTA: Spatial-Temporal Causal Adaptive Learning for Exemplar-Free Video Class-Incremental Learning [62.69917996026769]
  A class-incremental learning task requires learning and preserving both spatial appearance and temporal action involvement.<n>We propose a framework that equips separate adapters to learn new class patterns, accommodating the incremental information requirements unique to each class.<n>A causal compensation mechanism is proposed to reduce the conflicts during increment and memorization for between different types of information.
  arXiv  Detail & Related papers  (2025-01-13T11:34:55Z)
• Learning to Learn without Forgetting using Attention [5.6739565497512405]
  Continual learning (CL) refers to the ability to continually learn over time by accommodating new knowledge while retaining previously learned experience. Current machine learning methods are highly prone to overwrite previously learned patterns and thus forget past experience. Since hand-crafting effective update mechanisms is difficult, we propose meta-learning a transformer-based to enhance CL.
  arXiv  Detail & Related papers  (2024-08-06T14:25:23Z)
• Parametric Augmentation for Time Series Contrastive Learning [33.47157775532995]
  We create positive examples that assist the model in learning robust and discriminative representations. Usually, preset human intuition directs the selection of relevant data augmentations. We propose a contrastive learning framework with parametric augmentation, AutoTCL, which can be adaptively employed to support time series representation learning.
  arXiv  Detail & Related papers  (2024-02-16T03:51:14Z)
• Time Series Contrastive Learning with Information-Aware Augmentations [57.45139904366001]
  A key component of contrastive learning is to select appropriate augmentations imposing some priors to construct feasible positive samples. How to find the desired augmentations of time series data that are meaningful for given contrastive learning tasks and datasets remains an open question. We propose a new contrastive learning approach with information-aware augmentations, InfoTS, that adaptively selects optimal augmentations for time series representation learning.
  arXiv  Detail & Related papers  (2023-03-21T15:02:50Z)
• Multi-Task Self-Supervised Time-Series Representation Learning [3.31490164885582]
  Time-series representation learning can extract representations from data with temporal dynamics and sparse labels. We propose a new time-series representation learning method by combining the advantages of self-supervised tasks. We evaluate the proposed framework on three downstream tasks: time-series classification, forecasting, and anomaly detection.
  arXiv  Detail & Related papers  (2023-03-02T07:44:06Z)
• NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research [96.53307645791179]
  We introduce the Never-Ending VIsual-classification Stream (NEVIS'22), a benchmark consisting of a stream of over 100 visual classification tasks. Despite being limited to classification, the resulting stream has a rich diversity of tasks from OCR, to texture analysis, scene recognition, and so forth. Overall, NEVIS'22 poses an unprecedented challenge for current sequential learning approaches due to the scale and diversity of tasks.
  arXiv  Detail & Related papers  (2022-11-15T18:57:46Z)
• Contrastive Learning with Boosted Memorization [36.957895270908324]
  Self-supervised learning has achieved a great success in the representation learning of visual and textual data. Recent attempts to consider self-supervised long-tailed learning are made by rebalancing in the loss perspective or the model perspective. We propose a novel Boosted Contrastive Learning (BCL) method to enhance the long-tailed learning in the label-unaware context.
  arXiv  Detail & Related papers  (2022-05-25T11:54:22Z)
• Mixing Up Contrastive Learning: Self-Supervised Representation Learning for Time Series [22.376529167056376]
  We propose an unsupervised contrastive learning framework motivated from the perspective of label smoothing. The proposed approach uses a novel contrastive loss that naturally exploits a data augmentation scheme. Experiments demonstrate the framework's superior performance compared to other representation learning approaches.
  arXiv  Detail & Related papers  (2022-03-17T11:49:21Z)
• BERT WEAVER: Using WEight AVERaging to enable lifelong learning for transformer-based models in biomedical semantic search engines [49.75878234192369]
  We present WEAVER, a simple, yet efficient post-processing method that infuses old knowledge into the new model. We show that applying WEAVER in a sequential manner results in similar word embedding distributions as doing a combined training on all data at once.
  arXiv  Detail & Related papers  (2022-02-21T10:34:41Z)
• Adversarial Graph Contrastive Learning with Information Regularization [51.14695794459399]
  Contrastive learning is an effective method in graph representation learning. Data augmentation on graphs is far less intuitive and much harder to provide high-quality contrastive samples. We propose a simple but effective method, Adversarial Graph Contrastive Learning (ARIEL) It consistently outperforms the current graph contrastive learning methods in the node classification task over various real-world datasets.
  arXiv  Detail & Related papers  (2022-02-14T05:54:48Z)
• Contrastive Neural Processes for Self-Supervised Learning [1.8059331230167266]
  We propose a novel self-supervised learning framework that combines contrastive learning with neural processes. It relies on recent advances in neural processes to perform time series forecasting. Unlike previous self-supervised methods, our augmentation pipeline is task-agnostic, enabling our method to perform well across various applications.
  arXiv  Detail & Related papers  (2021-10-24T21:01:27Z)
• Continual Contrastive Self-supervised Learning for Image Classification [10.070132585425938]
  Self-supervise learning method shows tremendous potential on visual representation without any labeled data at scale. To improve the visual representation of self-supervised learning, larger and more varied data is needed. In this paper, we make the first attempt to implement the continual contrastive self-supervised learning by proposing a rehearsal method.
  arXiv  Detail & Related papers  (2021-07-05T03:53:42Z)
• Few-Cost Salient Object Detection with Adversarial-Paced Learning [95.0220555274653]
  This paper proposes to learn the effective salient object detection model based on the manual annotation on a few training images only. We name this task as the few-cost salient object detection and propose an adversarial-paced learning (APL)-based framework to facilitate the few-cost learning scenario.
  arXiv  Detail & Related papers  (2021-04-05T14:15:49Z)
• Composable Augmentation Encoding for Video Representation Learning [94.2358972764708]
  We focus on contrastive methods for self-supervised video representation learning. A common paradigm in contrastive learning is to construct positive pairs by sampling different data views for the same instance, with different data instances as negatives. We propose an 'augmentation aware' contrastive learning framework, where we explicitly provide a sequence of augmentation parameterisations. We show that our method encodes valuable information about specified spatial or temporal augmentation, and in doing so also achieve state-of-the-art performance on a number of video benchmarks.
  arXiv  Detail & Related papers  (2021-04-01T16:48:53Z)
• DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation [57.358212277226315]
  In imitation learning from observation IfO, a learning agent seeks to imitate a demonstrating agent using only observations of the demonstrated behavior without access to the control signals generated by the demonstrator. Recent methods based on adversarial imitation learning have led to state-of-the-art performance on IfO problems, but they typically suffer from high sample complexity due to a reliance on data-inefficient, model-free reinforcement learning algorithms. This issue makes them impractical to deploy in real-world settings, where gathering samples can incur high costs in terms of time, energy, and risk. We propose a more data-efficient IfO algorithm
  arXiv  Detail & Related papers  (2021-03-31T23:46:32Z)
• Visualising Deep Network's Time-Series Representations [93.73198973454944]
  Despite the popularisation of machine learning models, more often than not they still operate as black boxes with no insight into what is happening inside the model. In this paper, a method that addresses that issue is proposed, with a focus on visualising multi-dimensional time-series data. Experiments on a high-frequency stock market dataset show that the method provides fast and discernible visualisations.
  arXiv  Detail & Related papers  (2021-03-12T09:53:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.