B-MoE: A Body-Part-Aware Mixture-of-Experts "All Parts Matter" Approach to Micro-Action Recognition

• URL: http://arxiv.org/abs/2603.24245v1
• Date: Wed, 25 Mar 2026 12:33:11 GMT
• Title: B-MoE: A Body-Part-Aware Mixture-of-Experts "All Parts Matter" Approach to Micro-Action Recognition
• Abstract summary: B-MoE is a Body-part-aware Mixture-of-Experts framework designed to explicitly model the structured nature of human motion.<n>A cross-attention routing mechanism learns inter-region relationships and dynamically selects the most informative regions for each micro-action.<n>Experiments show consistent state-of-theart gains, with improvements in ambiguous, underrepresented, and low amplitude classes.
• Score: 5.793225171047889
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Micro-actions, fleeting and low-amplitude motions, such as glances, nods, or minor posture shifts, carry rich social meaning but remain difficult for current action recognition models to recognize due to their subtlety, short duration, and high inter-class ambiguity. In this paper, we introduce B-MoE, a Body-part-aware Mixture-of-Experts framework designed to explicitly model the structured nature of human motion. In B-MoE, each expert specializes in a distinct body region (head, body, upper limbs, lower limbs), and is based on the lightweight Macro-Micro Motion Encoder (M3E) that captures long-range contextual structure and fine-grained local motion. A cross-attention routing mechanism learns inter-region relationships and dynamically selects the most informative regions for each micro-action. B-MoE uses a dual-stream encoder that fuses these region-specific semantic cues with global motion features to jointly capture spatially localized cues and temporally subtle variations that characterize micro-actions. Experiments on three challenging benchmarks (MA-52, SocialGesture, and MPII-GroupInteraction) show consistent state-of-theart gains, with improvements in ambiguous, underrepresented, and low amplitude classes.

Related papers

• STAG: Spatio-temporal Evolving Structural Representation of Action Units for Micro-expression Recognition [0.7799711162530713]
  Micro-expression recognition is challenging due to subtle and short-lived facial muscle movements.<n>Existing methods rely heavily on apex-onset frames, overlook fine-grained inter-frame dynamics, and separately model spatial and temporal information.<n>We propose STAG, a dynamic ROI-AU-coupled spatial-temporal network that jointly models motion flow and adaptive facial connectivity.
  arXiv  Detail & Related papers  (2026-06-26T13:46:48Z)
• Micro-DualNet: Dual-Path Spatio-Temporal Network for Micro-Action Recognition [4.60357552784923]
  Micro-actions exhibit diverse characteristics where some are defined by spatial configurations while others manifest through temporal dynamics.<n>Existing methods that commit to single-temporal decomposition cannot accommodate this diversity.<n>We introduce entity-level adaptive routing where each body part learns its optimal processing preference, complemented by Mutual Action (MAC) loss loss enforces cross-path coherence.
  arXiv  Detail & Related papers  (2026-04-22T19:00:05Z)
• DEFT-LLM: Disentangled Expert Feature Tuning for Micro-Expression Recognition [16.903294278064667]
  We propose DEFT-LLM, which achieves semantic alignment by multi-expert disentanglement.<n>We first introduce Uni-MER, a motion-driven instruction designed to align text with local facial motion.<n>We then design an architecture with three experts to decouple facial dynamics into independent representations.
  arXiv  Detail & Related papers  (2025-11-14T04:21:24Z)
• Motion Matters: Motion-guided Modulation Network for Skeleton-based Micro-Action Recognition [26.997350207742034]
  Micro-Actions (MAs) are an important form of non-verbal communication in social interactions.<n>Existing methods in Micro-Action Recognition often overlook the inherent subtle changes in MAs.<n>We present a novel Motion-guided Modulation Network (MMN) that implicitly captures and modulates subtle motion cues.
  arXiv  Detail & Related papers  (2025-07-29T16:27:10Z)
• MOL: Joint Estimation of Micro-Expression, Optical Flow, and Landmark via Transformer-Graph-Style Convolution [46.600316142855334]
  Facial micro-expression recognition (MER) is a challenging problem, due to transient and subtle micro-expression (ME) actions.<n>We propose an end-to-end micro-action-aware deep learning framework with advantages from transformer, graph convolution, and vanilla convolution.<n>Our framework outperforms the state-of-the-art MER methods on CASME II, SAMM, and SMIC benchmarks.
  arXiv  Detail & Related papers  (2025-06-17T13:35:06Z)
• Neuron: Learning Context-Aware Evolving Representations for Zero-Shot Skeleton Action Recognition [64.56321246196859]
  We propose a novel dyNamically Evolving dUal skeleton-semantic syneRgistic framework.<n>We first construct the spatial-temporal evolving micro-prototypes and integrate dynamic context-aware side information.<n>We introduce the spatial compression and temporal memory mechanisms to guide the growth of spatial-temporal micro-prototypes.
  arXiv  Detail & Related papers  (2024-11-18T05:16:11Z)
• Adaptive Temporal Motion Guided Graph Convolution Network for Micro-expression Recognition [48.21696443824074]
  We propose a novel framework for micro-expression recognition, named the Adaptive Temporal Motion Guided Graph Convolution Network (ATM-GCN) Our framework excels at capturing temporal dependencies between frames across the entire clip, thereby enhancing micro-expression recognition at the clip level.
  arXiv  Detail & Related papers  (2024-06-13T10:57:24Z)
• MASA: Motion-aware Masked Autoencoder with Semantic Alignment for Sign Language Recognition [94.56755080185732]
  We propose a Motion-Aware masked autoencoder with Semantic Alignment (MASA) that integrates rich motion cues and global semantic information. Our framework can simultaneously learn local motion cues and global semantic features for comprehensive sign language representation.
  arXiv  Detail & Related papers  (2024-05-31T08:06:05Z)
• SpatioTemporal Focus for Skeleton-based Action Recognition [66.8571926307011]
  Graph convolutional networks (GCNs) are widely adopted in skeleton-based action recognition. We argue that the performance of recent proposed skeleton-based action recognition methods is limited by the following factors. Inspired by the recent attention mechanism, we propose a multi-grain contextual focus module, termed MCF, to capture the action associated relation information.
  arXiv  Detail & Related papers  (2022-03-31T02:45:24Z)
• Group Gated Fusion on Attention-based Bidirectional Alignment for Multimodal Emotion Recognition [63.07844685982738]
  This paper presents a new model named as Gated Bidirectional Alignment Network (GBAN), which consists of an attention-based bidirectional alignment network over LSTM hidden states. We empirically show that the attention-aligned representations outperform the last-hidden-states of LSTM significantly. The proposed GBAN model outperforms existing state-of-the-art multimodal approaches on the IEMOCAP dataset.
  arXiv  Detail & Related papers  (2022-01-17T09:46:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.