OMG-Bench: A New Challenging Benchmark for Skeleton-based Online Micro Hand Gesture Recognition

• URL: http://arxiv.org/abs/2512.16727v1
• Date: Thu, 18 Dec 2025 16:27:31 GMT
• Title: OMG-Bench: A New Challenging Benchmark for Skeleton-based Online Micro Hand Gesture Recognition
• Authors: Haochen Chang, Pengfei Ren, Buyuan Zhang, Da Li, Tianhao Han, Haoyang Zhang, Liang Xie, Hongbo Chen, Erwei Yin,
• Abstract summary: OMG-Bench is the first large-scale benchmark for skeleton-based online micro gesture recognition.<n>HMATr is an end-to-end framework that unifies gesture detection and classification by leveraging hierarchical memory banks.<n>Experiments show HMATr outperforms state-of-the-art methods by 7.6% in detection rate.
• Score: 15.24938776679168
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Online micro gesture recognition from hand skeletons is critical for VR/AR interaction but faces challenges due to limited public datasets and task-specific algorithms. Micro gestures involve subtle motion patterns, which make constructing datasets with precise skeletons and frame-level annotations difficult. To this end, we develop a multi-view self-supervised pipeline to automatically generate skeleton data, complemented by heuristic rules and expert refinement for semi-automatic annotation. Based on this pipeline, we introduce OMG-Bench, the first large-scale public benchmark for skeleton-based online micro gesture recognition. It features 40 fine-grained gesture classes with 13,948 instances across 1,272 sequences, characterized by subtle motions, rapid dynamics, and continuous execution. To tackle these challenges, we propose Hierarchical Memory-Augmented Transformer (HMATr), an end-to-end framework that unifies gesture detection and classification by leveraging hierarchical memory banks which store frame-level details and window-level semantics to preserve historical context. In addition, it employs learnable position-aware queries initialized from the memory to implicitly encode gesture positions and semantics. Experiments show that HMATr outperforms state-of-the-art methods by 7.6\% in detection rate, establishing a strong baseline for online micro gesture recognition. Project page: https://omg-bench.github.io/

Related papers

• DynaPURLS: Dynamic Refinement of Part-aware Representations for Skeleton-based Zero-Shot Action Recognition [51.80782323686666]
  We introduce textbfDynaPURLS, a unified framework that establishes robust, multi-scale visual-semantic correspondences.<n>Our framework leverages a large language model to generate hierarchical textual descriptions that encompass both global movements and local body-part dynamics.<n>Experiments on three large-scale benchmark datasets, including NTU RGB+D 60/120 and PKU-MMD, demonstrate that DynaPURLS significantly outperforms prior art.
  arXiv  Detail & Related papers  (2025-12-12T10:39:10Z)
• Micro-Expression Recognition via Fine-Grained Dynamic Perception [64.26947471761916]
  We develop a novel fine-grained dynamic perception (FDP) framework for facial micro-expression recognition (MER)<n>We rank frame-level features of a sequence of raw frames in chronological order, in which the rank process encodes the dynamic information of both ME appearances and motions.<n>Our method significantly outperforms the state-of-the-art MER methods, and works well for dynamic image construction.
  arXiv  Detail & Related papers  (2025-09-07T11:13:50Z)
• MAMMA: Markerless & Automatic Multi-Person Motion Action Capture [37.06717786024836]
  MAMMA is a markerless motion-capture pipeline that recovers SMPL-X parameters from multi-view video of two-person interaction sequences.<n>We introduce a method that predicts dense 2D surface landmarks conditioned on segmentation masks.<n>We demonstrate that our approach can handle complex person--person interaction and offers greater accuracy than existing methods.
  arXiv  Detail & Related papers  (2025-06-16T02:04:51Z)
• Skeleton2vec: A Self-supervised Learning Framework with Contextualized Target Representations for Skeleton Sequence [56.092059713922744]
  We show that using high-level contextualized features as prediction targets can achieve superior performance. Specifically, we propose Skeleton2vec, a simple and efficient self-supervised 3D action representation learning framework. Our proposed Skeleton2vec outperforms previous methods and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2024-01-01T12:08:35Z)
• Co-Speech Gesture Detection through Multi-Phase Sequence Labeling [3.924524252255593]
  We introduce a novel framework that reframes the task as a multi-phase sequence labeling problem. We evaluate our proposal on a large dataset of diverse co-speech gestures in task-oriented face-to-face dialogues.
  arXiv  Detail & Related papers  (2023-08-21T12:27:18Z)
• SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-supervised Skeleton Action Recognition [13.283178393519234]
  Self-supervised skeleton-based action recognition has attracted more attention. With utilizing the unlabeled data, more generalizable features can be learned to alleviate the overfitting problem. We propose a spatial-temporal masked autoencoder framework for self-supervised 3D skeleton-based action recognition.
  arXiv  Detail & Related papers  (2022-09-01T20:54:27Z)
• Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble [71.97020373520922]
  Sign language is commonly used by deaf or mute people to communicate. We propose a novel Multi-modal Framework with a Global Ensemble Model (GEM) for isolated Sign Language Recognition ( SLR) Our proposed SAM- SLR-v2 framework is exceedingly effective and achieves state-of-the-art performance with significant margins.
  arXiv  Detail & Related papers  (2021-10-12T16:57:18Z)
• SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild [62.450907796261646]
  Recognition of hand gestures can be performed directly from the stream of hand skeletons estimated by software. Despite the recent advancements in gesture and action recognition from skeletons, it is unclear how well the current state-of-the-art techniques can perform in a real-world scenario. This paper presents the results of the SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild contest.
  arXiv  Detail & Related papers  (2021-06-21T10:57:49Z)
• A Self-Supervised Gait Encoding Approach with Locality-Awareness for 3D Skeleton Based Person Re-Identification [65.18004601366066]
  Person re-identification (Re-ID) via gait features within 3D skeleton sequences is a newly-emerging topic with several advantages. This paper proposes a self-supervised gait encoding approach that can leverage unlabeled skeleton data to learn gait representations for person Re-ID.
  arXiv  Detail & Related papers  (2020-09-05T16:06:04Z)
• Gesture Recognition from Skeleton Data for Intuitive Human-Machine Interaction [0.6875312133832077]
  We propose an approach for segmentation and classification of dynamic gestures based on a set of handcrafted features. The method for gesture recognition applies a sliding window, which extracts information from both the spatial and temporal dimensions. At the end, the recognized gestures are used to interact with a collaborative robot.
  arXiv  Detail & Related papers  (2020-08-26T11:28:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.