An Ensemble of Knowledge Sharing Models for Dynamic Hand Gesture Recognition

• URL: http://arxiv.org/abs/2008.05732v1
• Date: Thu, 13 Aug 2020 07:37:27 GMT
• Title: An Ensemble of Knowledge Sharing Models for Dynamic Hand Gesture Recognition
• Authors: Kenneth Lai and Svetlana Yanushkevich
• Abstract summary: The focus of this paper is dynamic gesture recognition in the context of the interaction between humans and machines. We propose a model consisting of two sub-networks, a transformer and an ordered-neuron long-short-term-memory based recurrent neural network (RNN) The proposed ensemble of knowledge-sharing models exhibits an overall accuracy of 86.11% using only skeleton information.
• Score: 1.7158296436650335
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The focus of this paper is dynamic gesture recognition in the context of the interaction between humans and machines. We propose a model consisting of two sub-networks, a transformer and an ordered-neuron long-short-term-memory (ON-LSTM) based recurrent neural network (RNN). Each sub-network is trained to perform the task of gesture recognition using only skeleton joints. Since each sub-network extracts different types of features due to the difference in architecture, the knowledge can be shared between the sub-networks. Through knowledge distillation, the features and predictions from each sub-network are fused together into a new fusion classifier. In addition, a cyclical learning rate can be used to generate a series of models that are combined in an ensemble, in order to yield a more generalizable prediction. The proposed ensemble of knowledge-sharing models exhibits an overall accuracy of 86.11% using only skeleton information, as tested using the Dynamic Hand Gesture-14/28 dataset

Related papers

• Transfer Learning with Reconstruction Loss [12.906500431427716]
  This paper proposes a novel approach for model training by adding into the model an additional reconstruction stage associated with a new reconstruction loss. The proposed approach encourages the learned features to be general and transferable, and therefore can be readily used for efficient transfer learning. For numerical simulations, three applications are studied: transfer learning on classifying MNIST handwritten digits, the device-to-device wireless network power allocation, and the multiple-input-single-output network downlink beamforming and localization.
  arXiv  Detail & Related papers  (2024-03-31T00:22:36Z)
• Squeeze aggregated excitation network [0.0]
  Convolutional neural networks have spatial representations which read patterns in the vision tasks. We propose SaEnet, Squeeze aggregated excitation network, for learning global channelwise representation in between layers.
  arXiv  Detail & Related papers  (2023-08-25T12:30:48Z)
• Joint-bone Fusion Graph Convolutional Network for Semi-supervised Skeleton Action Recognition [65.78703941973183]
  We propose a novel correlation-driven joint-bone fusion graph convolutional network (CD-JBF-GCN) as an encoder and use a pose prediction head as a decoder. Specifically, the CD-JBF-GC can explore the motion transmission between the joint stream and the bone stream. The pose prediction based auto-encoder in the self-supervised training stage allows the network to learn motion representation from unlabeled data.
  arXiv  Detail & Related papers  (2022-02-08T16:03:15Z)
• Hierarchical Variational Memory for Few-shot Learning Across Domains [120.87679627651153]
  We introduce a hierarchical prototype model, where each level of the prototype fetches corresponding information from the hierarchical memory. The model is endowed with the ability to flexibly rely on features at different semantic levels if the domain shift circumstances so demand. We conduct thorough ablation studies to demonstrate the effectiveness of each component in our model.
  arXiv  Detail & Related papers  (2021-12-15T15:01:29Z)
• Multi-Scale Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition [140.18376685167857]
  A simple yet effective multi-scale semantics-guided neural network is proposed for skeleton-based action recognition. MS-SGN achieves the state-of-the-art performance on the NTU60, NTU120, and SYSU datasets.
  arXiv  Detail & Related papers  (2021-11-07T03:50:50Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)
• Gesture Recognition from Skeleton Data for Intuitive Human-Machine Interaction [0.6875312133832077]
  We propose an approach for segmentation and classification of dynamic gestures based on a set of handcrafted features. The method for gesture recognition applies a sliding window, which extracts information from both the spatial and temporal dimensions. At the end, the recognized gestures are used to interact with a collaborative robot.
  arXiv  Detail & Related papers  (2020-08-26T11:28:50Z)
• CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition [3.0255457622022486]
  We propose to combine the power of two deep learning techniques, the convolutional neural networks (CNN) and the recurrent neural networks (RNN) An overall accuracy of 85.46% is achieved on the dynamic hand gesture-14/28 dataset.
  arXiv  Detail & Related papers  (2020-07-22T10:25:19Z)
• Incremental Training of a Recurrent Neural Network Exploiting a Multi-Scale Dynamic Memory [79.42778415729475]
  We propose a novel incrementally trained recurrent architecture targeting explicitly multi-scale learning. We show how to extend the architecture of a simple RNN by separating its hidden state into different modules. We discuss a training algorithm where new modules are iteratively added to the model to learn progressively longer dependencies.
  arXiv  Detail & Related papers  (2020-06-29T08:35:49Z)
• Model Fusion via Optimal Transport [64.13185244219353]
  We present a layer-wise model fusion algorithm for neural networks. We show that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.
  arXiv  Detail & Related papers  (2019-10-12T22:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.