SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild
- URL: http://arxiv.org/abs/2106.10980v1
- Date: Mon, 21 Jun 2021 10:57:49 GMT
- Title: SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild
- Authors: Ariel Caputo, Andrea Giachetti, Simone Soso, Deborah Pintani, Andrea
D'Eusanio, Stefano Pini, Guido Borghi, Alessandro Simoni, Roberto Vezzani,
Rita Cucchiara, Andrea Ranieri, Franca Giannini, Katia Lupinetti, Marina
Monti, Mehran Maghoumi, Joseph J. LaViola Jr, Minh-Quan Le, Hai-Dang Nguyen,
Minh-Triet Tran
- Abstract summary: Recognition of hand gestures can be performed directly from the stream of hand skeletons estimated by software.
Despite the recent advancements in gesture and action recognition from skeletons, it is unclear how well the current state-of-the-art techniques can perform in a real-world scenario.
This paper presents the results of the SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild contest.
- Score: 62.450907796261646
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Gesture recognition is a fundamental tool to enable novel interaction
paradigms in a variety of application scenarios like Mixed Reality
environments, touchless public kiosks, entertainment systems, and more.
Recognition of hand gestures can be nowadays performed directly from the stream
of hand skeletons estimated by software provided by low-cost trackers
(Ultraleap) and MR headsets (Hololens, Oculus Quest) or by video processing
software modules (e.g. Google Mediapipe). Despite the recent advancements in
gesture and action recognition from skeletons, it is unclear how well the
current state-of-the-art techniques can perform in a real-world scenario for
the recognition of a wide set of heterogeneous gestures, as many benchmarks do
not test online recognition and use limited dictionaries. This motivated the
proposal of the SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in
the Wild. For this contest, we created a novel dataset with heterogeneous
gestures featuring different types and duration. These gestures have to be
found inside sequences in an online recognition scenario. This paper presents
the result of the contest, showing the performances of the techniques proposed
by four research groups on the challenging task compared with a simple baseline
method.
Related papers
- Zero-Shot Underwater Gesture Recognition [3.4078654008228924]
Hand gesture recognition allows humans to interact with machines non-verbally, which has a huge application in underwater exploration using autonomous underwater vehicles.
Recently, a new gesture-based language called CADDIAN has been devised for divers, and supervised learning methods have been applied to recognize the gestures with high accuracy.
In this work, we advocate the need for zero-shot underwater gesture recognition (ZSUGR), where the objective is to train a model with visual samples of gestures from a few seen'' classes only and transfer the gained knowledge at test time to recognize semantically-similar unseen gesture classes as well.
arXiv Detail & Related papers (2024-07-19T08:16:46Z) - Part-aware Unified Representation of Language and Skeleton for Zero-shot Action Recognition [57.97930719585095]
We introduce Part-aware Unified Representation between Language and Skeleton (PURLS) to explore visual-semantic alignment at both local and global scales.
Our approach is evaluated on various skeleton/language backbones and three large-scale datasets.
The results showcase the universality and superior performance of PURLS, surpassing prior skeleton-based solutions and standard baselines from other domains.
arXiv Detail & Related papers (2024-06-19T08:22:32Z) - SHREC 2022 Track on Online Detection of Heterogeneous Gestures [11.447098172408111]
This paper presents the outcomes of a contest organized to evaluate methods for the online recognition of heterogeneous gestures from sequences of 3D hand poses.
The dataset features continuous sequences of hand tracking data where the gestures are interleaved with non-significant motions.
The evaluation is based not only on the detection performances but also on the latency and the false positives, making it possible to understand the feasibility of practical interaction tools.
arXiv Detail & Related papers (2022-07-14T07:24:02Z) - The Gesture Authoring Space: Authoring Customised Hand Gestures for
Grasping Virtual Objects in Immersive Virtual Environments [81.5101473684021]
This work proposes a hand gesture authoring tool for object specific grab gestures allowing virtual objects to be grabbed as in the real world.
The presented solution uses template matching for gesture recognition and requires no technical knowledge to design and create custom tailored hand gestures.
The study showed that gestures created with the proposed approach are perceived by users as a more natural input modality than the others.
arXiv Detail & Related papers (2022-07-03T18:33:33Z) - Snapture -- A Novel Neural Architecture for Combined Static and Dynamic
Hand Gesture Recognition [19.320551882950706]
We propose a novel hybrid hand gesture recognition system.
Our architecture enables learning both static and dynamic gestures.
Our work contributes both to gesture recognition research and machine learning applications for non-verbal communication with robots.
arXiv Detail & Related papers (2022-05-28T11:12:38Z) - Muscle Vision: Real Time Keypoint Based Pose Classification of Physical
Exercises [52.77024349608834]
3D human pose recognition extrapolated from video has advanced to the point of enabling real-time software applications.
We propose a new machine learning pipeline and web interface that performs human pose recognition on a live video feed to detect when common exercises are performed and classify them accordingly.
arXiv Detail & Related papers (2022-03-23T00:55:07Z) - Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble [71.97020373520922]
Sign language is commonly used by deaf or mute people to communicate.
We propose a novel Multi-modal Framework with a Global Ensemble Model (GEM) for isolated Sign Language Recognition ( SLR)
Our proposed SAM- SLR-v2 framework is exceedingly effective and achieves state-of-the-art performance with significant margins.
arXiv Detail & Related papers (2021-10-12T16:57:18Z) - Relational Graph Learning on Visual and Kinematics Embeddings for
Accurate Gesture Recognition in Robotic Surgery [84.73764603474413]
We propose a novel online approach of multi-modal graph network (i.e., MRG-Net) to dynamically integrate visual and kinematics information.
The effectiveness of our method is demonstrated with state-of-the-art results on the public JIGSAWS dataset.
arXiv Detail & Related papers (2020-11-03T11:00:10Z) - Gesture Recognition from Skeleton Data for Intuitive Human-Machine
Interaction [0.6875312133832077]
We propose an approach for segmentation and classification of dynamic gestures based on a set of handcrafted features.
The method for gesture recognition applies a sliding window, which extracts information from both the spatial and temporal dimensions.
At the end, the recognized gestures are used to interact with a collaborative robot.
arXiv Detail & Related papers (2020-08-26T11:28:50Z) - 3D dynamic hand gestures recognition using the Leap Motion sensor and
convolutional neural networks [0.0]
We present a method for the recognition of a set of non-static gestures acquired through the Leap Motion sensor.
The acquired gesture information is converted in color images, where the variation of hand joint positions during the gesture are projected on a plane.
The classification of the gestures is performed using a deep Convolutional Neural Network (CNN)
arXiv Detail & Related papers (2020-03-03T11:05:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.