Improving Time Series Classification Algorithms Using Octave-Convolutional Layers

• URL: http://arxiv.org/abs/2109.13696v1
• Date: Tue, 28 Sep 2021 13:12:09 GMT
• Title: Improving Time Series Classification Algorithms Using Octave-Convolutional Layers
• Authors: Samuel Harford, Fazle Karim, Houshang Darabi
• Abstract summary: We experimentally show that by substituting convolutions with OctConv, we significantly improve accuracy for time series classification tasks. In addition, the updated ALSTM-OctFCN performs statistically the same as the top two time series classifers.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning models utilizing convolution layers have achieved state-of-the-art performance on univariate time series classification tasks. In this work, we propose improving CNN based time series classifiers by utilizing Octave Convolutions (OctConv) to outperform themselves. These network architectures include Fully Convolutional Networks (FCN), Residual Neural Networks (ResNets), LSTM-Fully Convolutional Networks (LSTM-FCN), and Attention LSTM-Fully Convolutional Networks (ALSTM-FCN). The proposed layers significantly improve each of these models with minimally increased network parameters. In this paper, we experimentally show that by substituting convolutions with OctConv, we significantly improve accuracy for time series classification tasks for most of the benchmark datasets. In addition, the updated ALSTM-OctFCN performs statistically the same as the top two time series classifers, TS-CHIEF and HIVE-COTE (both ensemble models). To further explore the impact of the OctConv layers, we perform ablation tests of the augmented model compared to their base model.

Related papers

• TCCT-Net: Two-Stream Network Architecture for Fast and Efficient Engagement Estimation via Behavioral Feature Signals [58.865901821451295]
  We present a novel two-stream feature fusion "Tensor-Convolution and Convolution-Transformer Network" (TCCT-Net) architecture. To better learn the meaningful patterns in the temporal-spatial domain, we design a "CT" stream that integrates a hybrid convolutional-transformer. In parallel, to efficiently extract rich patterns from the temporal-frequency domain, we introduce a "TC" stream that uses Continuous Wavelet Transform (CWT) to represent information in a 2D tensor form.
  arXiv  Detail & Related papers  (2024-04-15T06:01:48Z)
• Pushing the Efficiency Limit Using Structured Sparse Convolutions [82.31130122200578]
  We propose Structured Sparse Convolution (SSC), which leverages the inherent structure in images to reduce the parameters in the convolutional filter. We show that SSC is a generalization of commonly used layers (depthwise, groupwise and pointwise convolution) in efficient architectures'' Architectures based on SSC achieve state-of-the-art performance compared to baselines on CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet classification benchmarks.
  arXiv  Detail & Related papers  (2022-10-23T18:37:22Z)
• Towards Energy-Efficient, Low-Latency and Accurate Spiking LSTMs [1.7969777786551424]
  Spiking Neural Networks (SNNs) have emerged as an attractive-temporal computing paradigm vision for complex tasks. We propose an optimized spiking long short-term memory networks (LSTM) training framework that involves a novel. rev-to-SNN conversion framework, followed by SNN training. We evaluate our framework on sequential learning tasks including temporal M, Google Speech Commands (GSC) datasets, and UCI Smartphone on different LSTM architectures.
  arXiv  Detail & Related papers  (2022-10-23T04:10:27Z)
• Image Classification using Sequence of Pixels [3.04585143845864]
  This study compares sequential image classification methods based on recurrent neural networks. We describe methods based on Long-Short-Term memory(LSTM), bidirectional Long-Short-Term memory(BiLSTM) architectures, etc.
  arXiv  Detail & Related papers  (2022-09-23T09:42:44Z)
• Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks [10.014879130837912]
  We propose a symmetric multi-scale architecture called Circular Dilated Convolutional Neural Network (CDIL-CNN) Our model gives classification logits in all positions, and we can apply a simple ensemble learning to achieve a better decision.
  arXiv  Detail & Related papers  (2022-01-06T16:58:59Z)
• A Comparative Study of Detecting Anomalies in Time Series Data Using LSTM and TCN Models [2.007262412327553]
  This paper compares two prominent deep learning modeling techniques. The Recurrent Neural Network (RNN)-based Long Short-Term Memory (LSTM) and the convolutional Neural Network (CNN)-based Temporal Convolutional Networks (TCN) are compared.
  arXiv  Detail & Related papers  (2021-12-17T02:46:55Z)
• Train your classifier first: Cascade Neural Networks Training from upper layers to lower layers [54.47911829539919]
  We develop a novel top-down training method which can be viewed as an algorithm for searching for high-quality classifiers. We tested this method on automatic speech recognition (ASR) tasks and language modelling tasks. The proposed method consistently improves recurrent neural network ASR models on Wall Street Journal, self-attention ASR models on Switchboard, and AWD-LSTM language models on WikiText-2.
  arXiv  Detail & Related papers  (2021-02-09T08:19:49Z)
• ACDC: Weight Sharing in Atom-Coefficient Decomposed Convolution [57.635467829558664]
  We introduce a structural regularization across convolutional kernels in a CNN. We show that CNNs now maintain performance with dramatic reduction in parameters and computations.
  arXiv  Detail & Related papers  (2020-09-04T20:41:47Z)
• Automatic Remaining Useful Life Estimation Framework with Embedded Convolutional LSTM as the Backbone [5.927250637620123]
  We propose a new LSTM variant called embedded convolutional LSTM (E NeuralTM) In ETM a group of different 1D convolutions is embedded into the LSTM structure. Through this, the temporal information is preserved between and within windows. We show the superiority of our proposed ETM approach over the state-of-the-art approaches on several widely used benchmark data sets for RUL Estimation.
  arXiv  Detail & Related papers  (2020-08-10T08:34:20Z)
• The Heterogeneity Hypothesis: Finding Layer-Wise Differentiated Network Architectures [179.66117325866585]
  We investigate a design space that is usually overlooked, i.e. adjusting the channel configurations of predefined networks. We find that this adjustment can be achieved by shrinking widened baseline networks and leads to superior performance. Experiments are conducted on various networks and datasets for image classification, visual tracking and image restoration.
  arXiv  Detail & Related papers  (2020-06-29T17:59:26Z)
• Binarizing MobileNet via Evolution-based Searching [66.94247681870125]
  We propose a use of evolutionary search to facilitate the construction and training scheme when binarizing MobileNet. Inspired by one-shot architecture search frameworks, we manipulate the idea of group convolution to design efficient 1-Bit Convolutional Neural Networks (CNNs) Our objective is to come up with a tiny yet efficient binary neural architecture by exploring the best candidates of the group convolution.
  arXiv  Detail & Related papers  (2020-05-13T13:25:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.