Motion Reinforces Appearance: RGB-Skeleton Gated Residual Fusion for Micro-Gesture Online Recognition

• URL: http://arxiv.org/abs/2606.11645v1
• Date: Wed, 10 Jun 2026 04:12:36 GMT
• Title: Motion Reinforces Appearance: RGB-Skeleton Gated Residual Fusion for Micro-Gesture Online Recognition
• Abstract summary: We propose DyFADet+, which extends DyFADet into a dual-stream RGB-skeleton framework.<n>On the SMG dataset, our method achieves an F1 score of 40.88, ranking 2nd in the Micro-gesture Online Recognition track.
• Score: 18.736764781779208
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Micro-gesture analysis attracts increasing attention for inferring spontaneous emotion from subtle body movements. Micro-gesture online recognition, which localizes and classifies each gesture instance in untrimmed videos, is a core task in the 4th EI-MiGA-IJCAI Challenge. Compared with typical temporal action detection, MGR emphasizes the localization and classification of actions, requiring the model to output the start time, end time, and category of each micro-gesture. Moreover, since micro-gestures are highly spontaneous, relying solely on a single modality makes it difficult to capture the complete and accurate multi-modal cues. In this work, we propose DyFADet+, which extends DyFADet into a dual-stream RGB-skeleton framework. In our model, both modalities are projected into shared multi-scale temporal embeddings and fused through a gated residual module, which adaptively injects skeleton motion into the RGB representation rather than using naive concatenation. Finally, these fused features are decoded by a Dynamic TAD head for online classification and boundary regression. On the SMG dataset, our method achieves an F1 score of 40.88, ranking 2nd in the Micro-gesture Online Recognition track.

Related papers

• A Multi-Modal Framework with Cross-Subject Pseudo-Labeling and Semantic Alignment for Micro-Gesture Recognition [21.577609169548168]
  Micro-gestures (MGs) are spontaneous and subtle body movements that frequently convey hidden human emotions.<n> Recognizing MGs in untrimmed videos remains highly challenging due to their extremely low signal-to-noise ratio.<n>We propose a comprehensive multi-modal framework for Track 1 of the 4th MiGA-IJCAI Challenge.
  arXiv  Detail & Related papers  (2026-06-11T08:04:42Z)
• Micro-DualNet: Dual-Path Spatio-Temporal Network for Micro-Action Recognition [4.60357552784923]
  Micro-actions exhibit diverse characteristics where some are defined by spatial configurations while others manifest through temporal dynamics.<n>Existing methods that commit to single-temporal decomposition cannot accommodate this diversity.<n>We introduce entity-level adaptive routing where each body part learns its optimal processing preference, complemented by Mutual Action (MAC) loss loss enforces cross-path coherence.
  arXiv  Detail & Related papers  (2026-04-22T19:00:05Z)
• OMG-Bench: A New Challenging Benchmark for Skeleton-based Online Micro Hand Gesture Recognition [15.24938776679168]
  OMG-Bench is the first large-scale benchmark for skeleton-based online micro gesture recognition.<n>HMATr is an end-to-end framework that unifies gesture detection and classification by leveraging hierarchical memory banks.<n>Experiments show HMATr outperforms state-of-the-art methods by 7.6% in detection rate.
  arXiv  Detail & Related papers  (2025-12-18T16:27:31Z)
• X-ReID: Multi-granularity Information Interaction for Video-Based Visible-Infrared Person Re-Identification [79.37768038337971]
  We propose a novel cross-modality feature learning framework named X-ReID for VVI-ReID.<n> Specifically, we first propose a Cross-modality Prototype Collaboration (CPC)<n>Then, a Multi-granularity Information Interaction (MII) is designed, incorporating short-term interactions from adjacent frames, long-term cross-frame information fusion, and cross-modality feature alignment.
  arXiv  Detail & Related papers  (2025-11-22T07:57:15Z)
• FMANet: A Novel Dual-Phase Optical Flow Approach with Fusion Motion Attention Network for Robust Micro-expression Recognition [0.0]
  micro-expression recognition is challenging due to the difficulty of capturing subtle facial movements.<n>We introduce a comprehensive motion representation, which integrates motion dynamics from both micro-expression phases into a unified descriptor.<n>We then propose FMANet, a novel end-to-end neural network architecture that internalizes the dual-phase analysis and magnitude modulation into learnable modules.
  arXiv  Detail & Related papers  (2025-10-09T05:36:40Z)
• Micro-Expression Recognition via Fine-Grained Dynamic Perception [64.26947471761916]
  We develop a novel fine-grained dynamic perception (FDP) framework for facial micro-expression recognition (MER)<n>We rank frame-level features of a sequence of raw frames in chronological order, in which the rank process encodes the dynamic information of both ME appearances and motions.<n>Our method significantly outperforms the state-of-the-art MER methods, and works well for dynamic image construction.
  arXiv  Detail & Related papers  (2025-09-07T11:13:50Z)
• HyRSM++: Hybrid Relation Guided Temporal Set Matching for Few-shot Action Recognition [51.2715005161475]
  We propose a novel Hybrid Relation guided temporal Set Matching approach for few-shot action recognition. The core idea of HyRSM++ is to integrate all videos within the task to learn discriminative representations. We show that our method achieves state-of-the-art performance under various few-shot settings.
  arXiv  Detail & Related papers  (2023-01-09T13:32:50Z)
• SpatioTemporal Focus for Skeleton-based Action Recognition [66.8571926307011]
  Graph convolutional networks (GCNs) are widely adopted in skeleton-based action recognition. We argue that the performance of recent proposed skeleton-based action recognition methods is limited by the following factors. Inspired by the recent attention mechanism, we propose a multi-grain contextual focus module, termed MCF, to capture the action associated relation information.
  arXiv  Detail & Related papers  (2022-03-31T02:45:24Z)
• Slow-Fast Visual Tempo Learning for Video-based Action Recognition [78.3820439082979]
  Action visual tempo characterizes the dynamics and the temporal scale of an action. Previous methods capture the visual tempo either by sampling raw videos with multiple rates, or by hierarchically sampling backbone features. We propose a Temporal Correlation Module (TCM) to extract action visual tempo from low-level backbone features at single-layer remarkably.
  arXiv  Detail & Related papers  (2022-02-24T14:20:04Z)
• Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble [71.97020373520922]
  Sign language is commonly used by deaf or mute people to communicate. We propose a novel Multi-modal Framework with a Global Ensemble Model (GEM) for isolated Sign Language Recognition ( SLR) Our proposed SAM- SLR-v2 framework is exceedingly effective and achieves state-of-the-art performance with significant margins.
  arXiv  Detail & Related papers  (2021-10-12T16:57:18Z)
• Multi-modal Fusion for Single-Stage Continuous Gesture Recognition [45.19890687786009]
  We introduce a single-stage continuous gesture recognition framework, called Temporal Multi-Modal Fusion (TMMF) TMMF can detect and classify multiple gestures in a video via a single model. This approach learns the natural transitions between gestures and non-gestures without the need for a pre-processing segmentation step.
  arXiv  Detail & Related papers  (2020-11-10T07:09:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.