3D dynamic hand gestures recognition using the Leap Motion sensor and convolutional neural networks

• URL: http://arxiv.org/abs/2003.01450v3
• Date: Wed, 2 Sep 2020 15:13:22 GMT
• Title: 3D dynamic hand gestures recognition using the Leap Motion sensor and convolutional neural networks
• Authors: Katia Lupinetti, Andrea Ranieri, Franca Giannini, Marina Monti
• Abstract summary: We present a method for the recognition of a set of non-static gestures acquired through the Leap Motion sensor. The acquired gesture information is converted in color images, where the variation of hand joint positions during the gesture are projected on a plane. The classification of the gestures is performed using a deep Convolutional Neural Network (CNN)
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Defining methods for the automatic understanding of gestures is of paramount importance in many application contexts and in Virtual Reality applications for creating more natural and easy-to-use human-computer interaction methods. In this paper, we present a method for the recognition of a set of non-static gestures acquired through the Leap Motion sensor. The acquired gesture information is converted in color images, where the variation of hand joint positions during the gesture are projected on a plane and temporal information is represented with color intensity of the projected points. The classification of the gestures is performed using a deep Convolutional Neural Network (CNN). A modified version of the popular ResNet-50 architecture is adopted, obtained by removing the last fully connected layer and adding a new layer with as many neurons as the considered gesture classes. The method has been successfully applied to the existing reference dataset and preliminary tests have already been performed for the real-time recognition of dynamic gestures performed by users.

Related papers

• Agile gesture recognition for capacitive sensing devices: adapting on-the-job [55.40855017016652]
  We demonstrate a hand gesture recognition system that uses signals from capacitive sensors embedded into the etee hand controller. The controller generates real-time signals from each of the wearer five fingers. We use a machine learning technique to analyse the time series signals and identify three features that can represent 5 fingers within 500 ms.
  arXiv  Detail & Related papers  (2023-05-12T17:24:02Z)
• The Gesture Authoring Space: Authoring Customised Hand Gestures for Grasping Virtual Objects in Immersive Virtual Environments [81.5101473684021]
  This work proposes a hand gesture authoring tool for object specific grab gestures allowing virtual objects to be grabbed as in the real world. The presented solution uses template matching for gesture recognition and requires no technical knowledge to design and create custom tailored hand gestures. The study showed that gestures created with the proposed approach are perceived by users as a more natural input modality than the others.
  arXiv  Detail & Related papers  (2022-07-03T18:33:33Z)
• Snapture -- A Novel Neural Architecture for Combined Static and Dynamic Hand Gesture Recognition [19.320551882950706]
  We propose a novel hybrid hand gesture recognition system. Our architecture enables learning both static and dynamic gestures. Our work contributes both to gesture recognition research and machine learning applications for non-verbal communication with robots.
  arXiv  Detail & Related papers  (2022-05-28T11:12:38Z)
• Neural Maximum A Posteriori Estimation on Unpaired Data for Motion Deblurring [87.97330195531029]
  We propose a Neural Maximum A Posteriori (NeurMAP) estimation framework for training neural networks to recover blind motion information and sharp content from unpaired data. The proposed NeurMAP is an approach to existing deblurring neural networks, and is the first framework that enables training image deblurring networks on unpaired datasets.
  arXiv  Detail & Related papers  (2022-04-26T08:09:47Z)
• A deep-learning--based multimodal depth-aware dynamic hand gesture recognition system [5.458813674116228]
  We focus on dynamic hand gesture (DHG) recognition using depth quantized image hand skeleton joint points. In particular, we explore the effect of using depth-quantized features in CNN and Recurrent Neural Network (RNN) based multi-modal fusion networks.
  arXiv  Detail & Related papers  (2021-07-06T11:18:53Z)
• SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild [62.450907796261646]
  Recognition of hand gestures can be performed directly from the stream of hand skeletons estimated by software. Despite the recent advancements in gesture and action recognition from skeletons, it is unclear how well the current state-of-the-art techniques can perform in a real-world scenario. This paper presents the results of the SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild contest.
  arXiv  Detail & Related papers  (2021-06-21T10:57:49Z)
• Learning Compositional Representation for 4D Captures with Neural ODE [72.56606274691033]
  We introduce a compositional representation for 4D captures, that disentangles shape, initial state, and motion respectively. To model the motion, a neural Ordinary Differential Equation (ODE) is trained to update the initial state conditioned on the learned motion code. A decoder takes the shape code and the updated pose code to reconstruct 4D captures at each time stamp.
  arXiv  Detail & Related papers  (2021-03-15T10:55:55Z)
• Understanding the hand-gestures using Convolutional Neural Networks and Generative Adversial Networks [0.0]
  The system consists of three modules: real time hand tracking, training gesture and gesture recognition using Convolutional Neural Networks. It has been tested to the vocabulary of 36 gestures including the alphabets and digits, and results effectiveness of the approach.
  arXiv  Detail & Related papers  (2020-11-10T02:20:43Z)
• Gesture Recognition from Skeleton Data for Intuitive Human-Machine Interaction [0.6875312133832077]
  We propose an approach for segmentation and classification of dynamic gestures based on a set of handcrafted features. The method for gesture recognition applies a sliding window, which extracts information from both the spatial and temporal dimensions. At the end, the recognized gestures are used to interact with a collaborative robot.
  arXiv  Detail & Related papers  (2020-08-26T11:28:50Z)
• Hierarchical Contrastive Motion Learning for Video Action Recognition [100.9807616796383]
  We present hierarchical contrastive motion learning, a new self-supervised learning framework to extract effective motion representations from raw video frames. Our approach progressively learns a hierarchy of motion features that correspond to different abstraction levels in a network. Our motion learning module is lightweight and flexible to be embedded into various backbone networks.
  arXiv  Detail & Related papers  (2020-07-20T17:59:22Z)
• A Deep Learning Framework for Recognizing both Static and Dynamic Gestures [0.8602553195689513]
  We propose a unified framework that recognizes both static and dynamic gestures, using simple RGB vision (without depth sensing) We employ a pose-driven spatial attention strategy, which guides our proposed Static and Dynamic gestures Network - StaDNet. In a number of experiments, we show that the proposed approach surpasses the state-of-the-art results on the large-scale Chalearn 2016 dataset.
  arXiv  Detail & Related papers  (2020-06-11T10:39:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.