Online Hand Gesture Recognition Using 3D Convolutional Neural Networks

• URL: http://arxiv.org/abs/2605.23409v1
• Date: Fri, 22 May 2026 09:18:00 GMT
• Title: Online Hand Gesture Recognition Using 3D Convolutional Neural Networks
• Abstract summary: An online hand gesture recognition system is proposed, which is able to localize gestures in real-time video stream and recognize what these gestures are.<n>For the overall performance of the system, the best group can respond within three seconds and reach 37.5% Levenshtein accuracy on the homemade dataset.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In human computer interaction, real-time detection and classification of dynamic hand gestures is challenging as: 1) the system must run in a real-time video stream and there is no noticeable lag in response after performing a gesture; 2) there is a large difference in how people perform gestures, making recognition more difficult. In this paper, an online hand gesture recognition system is proposed, which is able to localize gestures in real-time video stream and recognize what these gestures are. To improve the robustness of the system, the sliding window approach is used to refine results from multiple windows. All of the models in my project are trained on Jester database, achieving 98+% accuracy for detector and 90+% accuracy for classifier. For the overall performance of the system, the best group can respond within three seconds and reach 37.5% Levenshtein accuracy on the homemade dataset. The project codes used in this work are publicly available.

Related papers

• Dynamic LIBRAS Gesture Recognition via CNN over Spatiotemporal Matrix Representation [0.0]
  This paper proposes a method for dynamic hand gesture recognition based on the composition of two models: the MediaPipe Hander, and a convolutional neural network (CNN)<n>The method is applied to recognition of RASLIB (Brazilian Sign Language) gestures for device control in a home automation system, covering 11 classes of static and dynamic gestures.<n>Tests achieved 95% accuracy under low-light conditions and 92% under normal lighting.
  arXiv  Detail & Related papers  (2026-03-26T19:37:28Z)
• WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild [53.288327629960364]
  We present a data-driven pipeline for efficient multi-hand reconstruction in the wild.<n>The proposed pipeline is composed of two components: a real-time fully convolutional hand localization and a high-fidelity transformer-based 3D hand reconstruction model.<n>Our approach outperforms previous methods in both efficiency and accuracy on popular 2D and 3D benchmarks.
  arXiv  Detail & Related papers  (2024-09-18T18:46:51Z)
• Agile gesture recognition for capacitive sensing devices: adapting on-the-job [55.40855017016652]
  We demonstrate a hand gesture recognition system that uses signals from capacitive sensors embedded into the etee hand controller. The controller generates real-time signals from each of the wearer five fingers. We use a machine learning technique to analyse the time series signals and identify three features that can represent 5 fingers within 500 ms.
  arXiv  Detail & Related papers  (2023-05-12T17:24:02Z)
• TempNet: Temporal Attention Towards the Detection of Animal Behaviour in Videos [63.85815474157357]
  We propose an efficient computer vision- and deep learning-based method for the detection of biological behaviours in videos. TempNet uses an encoder bridge and residual blocks to maintain model performance with a two-staged, spatial, then temporal, encoder. We demonstrate its application to the detection of sablefish (Anoplopoma fimbria) startle events.
  arXiv  Detail & Related papers  (2022-11-17T23:55:12Z)
• Design of Human Machine Interface through vision-based low-cost Hand Gesture Recognition system based on deep CNN [3.5665681694253903]
  A real-time hand gesture recognition system-based human-computer interface (HCI) is presented. The system consists of six stages: hand detection, (2) gesture segmentation, (3) use of six pre-trained CNN models by using the transfer-learning method, (4) building an interactive human-machine interface, (5) development of a gesture-controlled virtual mouse.
  arXiv  Detail & Related papers  (2022-07-07T06:50:08Z)
• On-device Real-time Hand Gesture Recognition [1.4658400971135652]
  We present an on-device real-time hand gesture recognition (HGR) system, which detects a set of predefined static gestures from a single RGB camera. We use MediaPipe Hands as the basis of the hand skeleton tracker, improve the keypoint accuracy, and add the estimation of 3D keypoints in a world metric space.
  arXiv  Detail & Related papers  (2021-10-29T18:33:25Z)
• Real-time Indian Sign Language (ISL) Recognition [0.45880283710344055]
  This paper presents a system which can recognise hand poses & gestures from the Indian Sign Language (ISL) in real-time. The existing solutions either provide relatively low accuracy or do not work in real-time. It can identify 33 hand poses and some gestures from the ISL.
  arXiv  Detail & Related papers  (2021-08-24T21:49:21Z)
• HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks [71.09275975580009]
  HandVoxNet++ is a voxel-based deep network with 3D and graph convolutions trained in a fully supervised manner. HandVoxNet++ relies on two hand shape representations. The first one is the 3D voxelized grid of hand shape, which does not preserve the mesh topology. We combine the advantages of both representations by aligning the hand surface to the voxelized hand shape either with a new neural Graph-Convolutions-based Mesh Registration (GCN-MeshReg) or classical segment-wise Non-Rigid Gravitational Approach (NRGA++) which
  arXiv  Detail & Related papers  (2021-07-02T17:59:54Z)
• SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild [62.450907796261646]
  Recognition of hand gestures can be performed directly from the stream of hand skeletons estimated by software. Despite the recent advancements in gesture and action recognition from skeletons, it is unclear how well the current state-of-the-art techniques can perform in a real-world scenario. This paper presents the results of the SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild contest.
  arXiv  Detail & Related papers  (2021-06-21T10:57:49Z)
• 2D or not 2D? Adaptive 3D Convolution Selection for Efficient Video Recognition [84.697097472401]
  We introduce Ada3D, a conditional computation framework that learns instance-specific 3D usage policies to determine frames and convolution layers to be used in a 3D network. We demonstrate that our method achieves similar accuracies to state-of-the-art 3D models while requiring 20%-50% less computation across different datasets.
  arXiv  Detail & Related papers  (2020-12-29T21:40:38Z)
• Understanding the hand-gestures using Convolutional Neural Networks and Generative Adversial Networks [0.0]
  The system consists of three modules: real time hand tracking, training gesture and gesture recognition using Convolutional Neural Networks. It has been tested to the vocabulary of 36 gestures including the alphabets and digits, and results effectiveness of the approach.
  arXiv  Detail & Related papers  (2020-11-10T02:20:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.