Model Selection with a Shapelet-based Distance Measure for Multi-source Transfer Learning in Time Series Classification

• URL: http://arxiv.org/abs/2409.20005v1
• Date: Mon, 30 Sep 2024 06:57:30 GMT
• Title: Model Selection with a Shapelet-based Distance Measure for Multi-source Transfer Learning in Time Series Classification
• Abstract summary: We propose a novel method of selecting and using multiple datasets for transfer learning for time series classification. Our method combines multiple datasets as one source dataset for pre-training neural networks. We demonstrate that it is possible to increase the performance of temporal convolutional neural networks (CNN) on time series datasets.
• Score: 6.875312133832079
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transfer learning is a common practice that alleviates the need for extensive data to train neural networks. It is performed by pre-training a model using a source dataset and fine-tuning it for a target task. However, not every source dataset is appropriate for each target dataset, especially for time series. In this paper, we propose a novel method of selecting and using multiple datasets for transfer learning for time series classification. Specifically, our method combines multiple datasets as one source dataset for pre-training neural networks. Furthermore, for selecting multiple sources, our method measures the transferability of datasets based on shapelet discovery for effective source selection. While traditional transferability measures require considerable time for pre-training all the possible sources for source selection of each possible architecture, our method can be repeatedly used for every possible architecture with a single simple computation. Using the proposed method, we demonstrate that it is possible to increase the performance of temporal convolutional neural networks (CNN) on time series datasets.

Related papers

• Multi-source Transfer Learning of Time Series with a Shapelet-based Distance Measure [3.7814216736076425]
  This paper introduces a training-free source selection method named Shapelet Matching.<n>Shapelet Matching quantifies dataset similarity by comparing the extracted sets of shapelets.<n>We select several source datasets based on their shapelet-based similarity scores, combine them into a single multi-source dataset, and use this aggregated dataset for pre-training.
  arXiv  Detail & Related papers  (2026-09-14T07:23:19Z)
• Private Training & Data Generation by Clustering Embeddings [74.00687214400021]
  Differential privacy (DP) provides a robust framework for protecting individual data.<n>We introduce a novel principled method for DP synthetic image embedding generation.<n> Empirically, a simple two-layer neural network trained on synthetically generated embeddings achieves state-of-the-art (SOTA) classification accuracy.
  arXiv  Detail & Related papers  (2025-06-20T00:17:14Z)
• BeST -- A Novel Source Selection Metric for Transfer Learning [35.32994166809785]
  We develop a novel task-similarity metric (BeST) to identify the most transferrable source(s) for a given task. Our metric can provide significant computational savings for transfer learning from a selection of a large number of possible source models.
  arXiv  Detail & Related papers  (2025-01-19T03:58:05Z)
• Swift Cross-Dataset Pruning: Enhancing Fine-Tuning Efficiency in Natural Language Understanding [2.379669478864599]
  Current cross-dataset pruning techniques for fine-tuning often rely on computationally expensive sample ranking processes. We propose Swift Cross-Dataset Pruning (SCDP), which uses TF-IDF embeddings with geometric median to rapidly evaluate sample importance. Experimental results on six diverse datasets demonstrate the effectiveness of our method, spanning various tasks and scales.
  arXiv  Detail & Related papers  (2025-01-05T03:52:04Z)
• Adapt-$\infty$: Scalable Lifelong Multimodal Instruction Tuning via Dynamic Data Selection [89.42023974249122]
  Adapt-$infty$ is a new multi-way and adaptive data selection approach for Lifelong Instruction Tuning. We construct pseudo-skill clusters by grouping gradient-based sample vectors. We select the best-performing data selector for each skill cluster from a pool of selector experts.
  arXiv  Detail & Related papers  (2024-10-14T15:48:09Z)
• Toward a Foundation Model for Time Series Data [34.1973242428317]
  A foundation model is a machine learning model trained on a large and diverse set of data. We develop an effective time series foundation model by leveraging unlabeled samples from multiple domains.
  arXiv  Detail & Related papers  (2023-10-05T21:44:50Z)
• Towards Free Data Selection with General-Purpose Models [71.92151210413374]
  A desirable data selection algorithm can efficiently choose the most informative samples to maximize the utility of limited annotation budgets. Current approaches, represented by active learning methods, typically follow a cumbersome pipeline that iterates the time-consuming model training and batch data selection repeatedly. FreeSel bypasses the heavy batch selection process, achieving a significant improvement in efficiency and being 530x faster than existing active learning methods.
  arXiv  Detail & Related papers  (2023-09-29T15:50:14Z)
• Iterative self-transfer learning: A general methodology for response time-history prediction based on small dataset [0.0]
  An iterative self-transfer learningmethod for training neural networks based on small datasets is proposed in this study. The results show that the proposed method can improve the model performance by near an order of magnitude on small datasets.
  arXiv  Detail & Related papers  (2023-06-14T18:48:04Z)
• Towards General and Efficient Active Learning [20.888364610175987]
  Active learning aims to select the most informative samples to exploit limited annotation budgets. We propose a novel general and efficient active learning (GEAL) method in this paper. Our method can conduct data selection processes on different datasets with a single-pass inference of the same model.
  arXiv  Detail & Related papers  (2021-12-15T08:35:28Z)
• Single-dataset Experts for Multi-dataset Question Answering [6.092171111087768]
  We train a network on multiple datasets to generalize and transfer better to new datasets. Our approach is to model multi-dataset question answering with a collection of single-dataset experts. Simple methods based on parameter-averaging lead to better zero-shot generalization and few-shot transfer performance.
  arXiv  Detail & Related papers  (2021-09-28T17:08:22Z)
• XMixup: Efficient Transfer Learning with Auxiliary Samples by Cross-domain Mixup [60.07531696857743]
  Cross-domain Mixup (XMixup) improves the multitask paradigm for deep transfer learning. XMixup selects the auxiliary samples from the source dataset and augments training samples via the simple mixup strategy. Experiment results show that XMixup improves the accuracy by 1.9% on average.
  arXiv  Detail & Related papers  (2020-07-20T16:42:29Z)
• Pre-Trained Models for Heterogeneous Information Networks [57.78194356302626]
  We propose a self-supervised pre-training and fine-tuning framework, PF-HIN, to capture the features of a heterogeneous information network. PF-HIN consistently and significantly outperforms state-of-the-art alternatives on each of these tasks, on four datasets.
  arXiv  Detail & Related papers  (2020-07-07T03:36:28Z)
• DeGAN : Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier [58.979104709647295]
  We bridge the gap between the abundance of available data and lack of relevant data, for the future learning tasks of a trained network. We use the available data, that may be an imbalanced subset of the original training dataset, or a related domain dataset, to retrieve representative samples. We demonstrate that data from a related domain can be leveraged to achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2019-12-27T02:05:45Z)
• Model Fusion via Optimal Transport [64.13185244219353]
  We present a layer-wise model fusion algorithm for neural networks. We show that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.
  arXiv  Detail & Related papers  (2019-10-12T22:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.