Look Into the LITE in Deep Learning for Time Series Classification

• URL: http://arxiv.org/abs/2409.02869v2
• Date: Thu, 23 Jan 2025 17:15:53 GMT
• Title: Look Into the LITE in Deep Learning for Time Series Classification
• Abstract summary: Deep learning models have been shown to be a powerful solution for Time Series Classification.<n>We present a new architecture for TSC, the Light Inception with boosTing tEchnique (LITE) with only 2.34% of the number of parameters of the state-of-the-art InceptionTime model.<n>The LITE architecture, trained on the UCR, is 2.78 times faster than InceptionTime and consumes 2.79 times less CO2 and power.
• Score: 6.708543240320757
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning models have been shown to be a powerful solution for Time Series Classification (TSC). State-of-the-art architectures, while producing promising results on the UCR and the UEA archives , present a high number of trainable parameters. This can lead to long training with high CO2 emission, power consumption and possible increase in the number of FLoating-point Operation Per Second (FLOPS). In this paper, we present a new architecture for TSC, the Light Inception with boosTing tEchnique (LITE) with only 2.34% of the number of parameters of the state-of-the-art InceptionTime model, while preserving performance. This architecture, with only 9, 814 trainable parameters due to the usage of DepthWise Separable Convolutions (DWSC), is boosted by three techniques: multiplexing, custom filters, and dilated convolution. The LITE architecture, trained on the UCR, is 2.78 times faster than InceptionTime and consumes 2.79 times less CO2 and power. To evaluate the performance of the proposed architecture on multivariate time series data, we adapt LITE to handle multivariate time series, we call this version LITEMV. To bring theory into application, we also conducted experiments using LITEMV on multivariate time series representing human rehabilitation movements, showing that LITEMV not only is the most efficient model but also the best performing for this application on the Kimore dataset, a skeleton based human rehabilitation exercises dataset. Moreover, to address the interpretability of LITEMV, we present a study using Class Activation Maps to understand the classification decision taken by the model during evaluation.

Related papers

• Enhancing deep learning models for time series classification via knowledge distillation [0.49387059996366106]
  Knowledge Distillation (KD) addresses this by transferring knowledge from a large teacher model to a smaller, more efficient student model.<n>We evaluate our approach on UCR Archive, the largest benchmark repository of time series datasets.
  arXiv  Detail & Related papers  (2026-07-07T20:51:22Z)
• Is Temporal Prompting All We Need For Limited Labeled Action Recognition? [11.47868206641396]
  We present TP-CLIP, an adaptation of CLIP that leverages temporal visual prompting for temporal adaptation without modifying the core CLIP architecture. TP-CLIP efficiently integrates into the CLIP architecture, leveraging its pre-trained capabilities for video data.
  arXiv  Detail & Related papers  (2025-04-02T16:50:28Z)
• Exploring Kolmogorov-Arnold Networks for Interpretable Time Series Classification [4.489167091323233]
  Kolmogorov-Arnold Networks (KANs) have been proposed as a more interpretable alternative to deep learning.<n>This paper explores the KAN architecture for time series classification utilising 117 datasets from UCR benchmark archive.
  arXiv  Detail & Related papers  (2024-11-22T13:01:36Z)
• SONNET: Enhancing Time Delay Estimation by Leveraging Simulated Audio [17.811771707446926]
  We show that learning based methods can, even based on synthetic data, significantly outperform GCC-PHAT on novel real world data. We provide our trained model, SONNET, which is runnable in real-time and works on novel data out of the box for many real data applications.
  arXiv  Detail & Related papers  (2024-11-20T10:23:21Z)
• The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute [66.84421705029624]
  We introduce an experimental protocol that enables model comparisons based on equivalent compute, measured in accelerator hours. We pre-process an existing large, diverse, and high-quality dataset of books that surpasses existing academic benchmarks in quality, diversity, and document length. This work also provides two baseline models: a feed-forward model derived from the GPT-2 architecture and a recurrent model in the form of a novel LSTM with ten-fold throughput.
  arXiv  Detail & Related papers  (2023-09-20T10:31:17Z)
• FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification [53.55504611255664]
  FormerTime is a hierarchical representation model for improving the classification capacity for the multivariate time series classification task. It exhibits three aspects of merits: (1) learning hierarchical multi-scale representations from time series data, (2) inheriting the strength of both transformers and convolutional networks, and (3) tacking the efficiency challenges incurred by the self-attention mechanism.
  arXiv  Detail & Related papers  (2023-02-20T07:46:14Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• An Experimental Review on Deep Learning Architectures for Time Series Forecasting [0.0]
  We provide the most extensive deep learning study for time series forecasting. Among all studied models, the results show that long short-term memory (LSTM) and convolutional networks (CNN) are the best alternatives. CNNs achieve comparable performance with less variability of results under different parameter configurations, while also being more efficient.
  arXiv  Detail & Related papers  (2021-03-22T17:58:36Z)
• Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data with Spatial Information [52.635997570873194]
  This work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to process complex multi-dimensional time series data with spatial information. The proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.
  arXiv  Detail & Related papers  (2021-01-12T20:08:18Z)
• On the performance of deep learning models for time series classification in streaming [0.0]
  This work is to assess the performance of different types of deep architectures for data streaming classification. We evaluate models such as multi-layer perceptrons, recurrent, convolutional and temporal convolutional neural networks over several time-series datasets.
  arXiv  Detail & Related papers  (2020-03-05T11:41:29Z)
• Convolutional Tensor-Train LSTM for Spatio-temporal Learning [116.24172387469994]
  We propose a higher-order LSTM model that can efficiently learn long-term correlations in the video sequence. This is accomplished through a novel tensor train module that performs prediction by combining convolutional features across time. Our results achieve state-of-the-art performance-art in a wide range of applications and datasets.
  arXiv  Detail & Related papers  (2020-02-21T05:00:01Z)
• Lipreading using Temporal Convolutional Networks [57.41253104365274]
  Current model for recognition of isolated words in-the-wild consists of a residual network and Bi-directional Gated Recurrent Unit layers. We address the limitations of this model and we propose changes which further improve its performance. Our proposed model results in an absolute improvement of 1.2% and 3.2%, respectively, in these datasets.
  arXiv  Detail & Related papers  (2020-01-23T17:49:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.