Dynamic LIBRAS Gesture Recognition via CNN over Spatiotemporal Matrix Representation

• URL: http://arxiv.org/abs/2603.25863v1
• Date: Thu, 26 Mar 2026 19:37:28 GMT
• Title: Dynamic LIBRAS Gesture Recognition via CNN over Spatiotemporal Matrix Representation
• Abstract summary: This paper proposes a method for dynamic hand gesture recognition based on the composition of two models: the MediaPipe Hander, and a convolutional neural network (CNN)<n>The method is applied to recognition of RASLIB (Brazilian Sign Language) gestures for device control in a home automation system, covering 11 classes of static and dynamic gestures.<n>Tests achieved 95% accuracy under low-light conditions and 92% under normal lighting.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper proposes a method for dynamic hand gesture recognition based on the composition of two models: the MediaPipe Hand Landmarker, responsible for extracting 21 skeletal keypoints of the hand, and a convolutional neural network (CNN) trained to classify gestures from a spatiotemporal matrix representation of dimensions 90 by 21 of those keypoints. The method is applied to the recognition of LIBRAS (Brazilian Sign Language) gestures for device control in a home automation system, covering 11 classes of static and dynamic gestures. For real-time inference, a sliding window with temporal frame triplication is used, enabling continuous recognition without recurrent networks. Tests achieved 95\% accuracy under low-light conditions and 92\% under normal lighting. The results indicate that the approach is effective, although systematic experiments with greater user diversity are needed for a more thorough evaluation of generalization.

Related papers

• Design and Validation of a Lightweight 1D CNN for Affective Touch Classification in Soft Plush Companions [59.49967357689445]
  Soft, sensorized companions offer a physically safe and emotionally intuitive interface for socially assistive technologies.<n>This study presents a complete open-source dilated real-based framework for the development and validation of deep learning models for affective touch recognition in soft interactive companions.
  arXiv  Detail & Related papers  (2026-04-16T10:20:05Z)
• Towards Arbitrary Motion Completing via Hierarchical Continuous Representation [64.6525112550758]
  We propose a novel parametric activation-induced hierarchical implicit representation framework, called NAME, based on Implicit Representations (INRs)<n>Our method introduces a hierarchical temporal encoding mechanism that extracts features from motion sequences at multiple temporal scales, enabling effective capture of intricate temporal patterns.
  arXiv  Detail & Related papers  (2025-12-24T14:07:04Z)
• Accurate online action and gesture recognition system using detectors and Deep SPD Siamese Networks [4.667044415202222]
  We propose an online recognition system for skeleton sequence streaming composed from two main components: a detector and a classifier.<n>The detector is flexible and able to identify the kinetic state continuously.<n>We conduct extensive experiments on both hand gesture and body action recognition benchmarks to prove the accuracy of our online recognition system.
  arXiv  Detail & Related papers  (2025-11-07T14:09:43Z)
• Skeleton2vec: A Self-supervised Learning Framework with Contextualized Target Representations for Skeleton Sequence [56.092059713922744]
  We show that using high-level contextualized features as prediction targets can achieve superior performance. Specifically, we propose Skeleton2vec, a simple and efficient self-supervised 3D action representation learning framework. Our proposed Skeleton2vec outperforms previous methods and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2024-01-01T12:08:35Z)
• Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognition [54.23513799338309]
  We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition. Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
  arXiv  Detail & Related papers  (2022-09-21T02:33:07Z)
• Towards Domain-Independent and Real-Time Gesture Recognition Using mmWave Signal [11.76969975145963]
  DI-Gesture is a domain-independent and real-time mmWave gesture recognition system. In real-time scenario, the accuracy of DI-Gesutre reaches over 97% with average inference time of 2.87ms.
  arXiv  Detail & Related papers  (2021-11-11T13:28:28Z)
• Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble [71.97020373520922]
  Sign language is commonly used by deaf or mute people to communicate. We propose a novel Multi-modal Framework with a Global Ensemble Model (GEM) for isolated Sign Language Recognition ( SLR) Our proposed SAM- SLR-v2 framework is exceedingly effective and achieves state-of-the-art performance with significant margins.
  arXiv  Detail & Related papers  (2021-10-12T16:57:18Z)
• HAN: An Efficient Hierarchical Self-Attention Network for Skeleton-Based Gesture Recognition [73.64451471862613]
  We propose an efficient hierarchical self-attention network (HAN) for skeleton-based gesture recognition. Joint self-attention module is used to capture spatial features of fingers, the finger self-attention module is designed to aggregate features of the whole hand. Experiments show that our method achieves competitive results on three gesture recognition datasets with much lower computational complexity.
  arXiv  Detail & Related papers  (2021-06-25T02:15:53Z)
• SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild [62.450907796261646]
  Recognition of hand gestures can be performed directly from the stream of hand skeletons estimated by software. Despite the recent advancements in gesture and action recognition from skeletons, it is unclear how well the current state-of-the-art techniques can perform in a real-world scenario. This paper presents the results of the SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild contest.
  arXiv  Detail & Related papers  (2021-06-21T10:57:49Z)
• Understanding the hand-gestures using Convolutional Neural Networks and Generative Adversial Networks [0.0]
  The system consists of three modules: real time hand tracking, training gesture and gesture recognition using Convolutional Neural Networks. It has been tested to the vocabulary of 36 gestures including the alphabets and digits, and results effectiveness of the approach.
  arXiv  Detail & Related papers  (2020-11-10T02:20:43Z)
• Gesture Recognition from Skeleton Data for Intuitive Human-Machine Interaction [0.6875312133832077]
  We propose an approach for segmentation and classification of dynamic gestures based on a set of handcrafted features. The method for gesture recognition applies a sliding window, which extracts information from both the spatial and temporal dimensions. At the end, the recognized gestures are used to interact with a collaborative robot.
  arXiv  Detail & Related papers  (2020-08-26T11:28:50Z)
• 3D dynamic hand gestures recognition using the Leap Motion sensor and convolutional neural networks [0.0]
  We present a method for the recognition of a set of non-static gestures acquired through the Leap Motion sensor. The acquired gesture information is converted in color images, where the variation of hand joint positions during the gesture are projected on a plane. The classification of the gestures is performed using a deep Convolutional Neural Network (CNN)
  arXiv  Detail & Related papers  (2020-03-03T11:05:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.