SMA-STN: Segmented Movement-Attending Spatiotemporal Network forMicro-Expression Recognition

• URL: http://arxiv.org/abs/2010.09342v1
• Date: Mon, 19 Oct 2020 09:23:24 GMT
• Title: SMA-STN: Segmented Movement-Attending Spatiotemporal Network forMicro-Expression Recognition
• Abstract summary: This paper proposes a segmented movement-attending network (SMA-STN) to reveal subtle movement changes visually in an efficient way. Extensive experiments on three widely used benchmarks, i.e., CALoss II, SAMM, and SHIC, show that the proposed SMA-STN achieves better MER performance than other state-of-the-art methods.
• Score: 20.166205708651194
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Correctly perceiving micro-expression is difficult since micro-expression is an involuntary, repressed, and subtle facial expression, and efficiently revealing the subtle movement changes and capturing the significant segments in a micro-expression sequence is the key to micro-expression recognition (MER). To handle the crucial issue, in this paper, we firstly propose a dynamic segmented sparse imaging module (DSSI) to compute dynamic images as local-global spatiotemporal descriptors under a unique sampling protocol, which reveals the subtle movement changes visually in an efficient way. Secondly, a segmented movement-attending spatiotemporal network (SMA-STN) is proposed to further unveil imperceptible small movement changes, which utilizes a spatiotemporal movement-attending module (STMA) to capture long-distance spatial relation for facial expression and weigh temporal segments. Besides, a deviation enhancement loss (DE-Loss) is embedded in the SMA-STN to enhance the robustness of SMA-STN to subtle movement changes in feature level. Extensive experiments on three widely used benchmarks, i.e., CASME II, SAMM, and SHIC, show that the proposed SMA-STN achieves better MER performance than other state-of-the-art methods, which proves that the proposed method is effective to handle the challenging MER problem.

Related papers

• STAG: Spatio-temporal Evolving Structural Representation of Action Units for Micro-expression Recognition [0.7799711162530713]
  Micro-expression recognition is challenging due to subtle and short-lived facial muscle movements.<n>Existing methods rely heavily on apex-onset frames, overlook fine-grained inter-frame dynamics, and separately model spatial and temporal information.<n>We propose STAG, a dynamic ROI-AU-coupled spatial-temporal network that jointly models motion flow and adaptive facial connectivity.
  arXiv  Detail & Related papers  (2026-06-26T13:46:48Z)
• Improving Micro-Expression Recognition with Phase-Aware Temporal Augmentation [0.0]
  Micro-expressions (MEs) are brief, involuntary facial movements that reveal genuine emotions, typically lasting less than half a second.<n>Deep learning has enabled significant advances in micro-expression recognition (MER), but its effectiveness is limited by the scarcity of annotated ME datasets.<n>This paper proposes a phase-aware temporal augmentation method based on dynamic image.
  arXiv  Detail & Related papers  (2025-10-17T09:20:51Z)
• DIANet: A Phase-Aware Dual-Stream Network for Micro-Expression Recognition via Dynamic Images [0.0]
  Micro-expressions are brief, involuntary facial movements that typically last less than half a second and often reveal genuine emotions.<n>This paper proposes a novel dual-stream framework, DIANet, which leverages phase-aware dynamic images.<n>Experiments conducted on three benchmark MER datasets demonstrate that the proposed method consistently outperforms conventional single-phase DI-based approaches.
  arXiv  Detail & Related papers  (2025-10-14T07:15:29Z)
• MPT: Motion Prompt Tuning for Micro-Expression Recognition [47.62949098749473]
  This paper introduces Motion Prompt Tuning (MPT) as a novel approach to adapting pre-training models for micro-expression recognition (MER)<n>MPT represents a pioneering method for subtle motion prompt tuning. Particularly, we introduce motion prompt generation, including motion magnification and Gaussian tokenization, to extract subtle motions as prompts for LMs.<n>Extensive experiments conducted on three widely used MER datasets demonstrate that our proposed MPT consistently surpasses state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-08-13T02:57:43Z)
• Motion Matters: Motion-guided Modulation Network for Skeleton-based Micro-Action Recognition [26.997350207742034]
  Micro-Actions (MAs) are an important form of non-verbal communication in social interactions.<n>Existing methods in Micro-Action Recognition often overlook the inherent subtle changes in MAs.<n>We present a novel Motion-guided Modulation Network (MMN) that implicitly captures and modulates subtle motion cues.
  arXiv  Detail & Related papers  (2025-07-29T16:27:10Z)
• Inference-Time Gaze Refinement for Micro-Expression Recognition: Enhancing Event-Based Eye Tracking with Motion-Aware Post-Processing [2.5465367830324905]
  Event-based eye tracking holds significant promise for fine-grained cognitive state inference.<n>We introduce a model-agnostic, inference-time refinement framework to enhance the output of existing event-based gaze estimation models.
  arXiv  Detail & Related papers  (2025-06-14T14:48:11Z)
• Adaptive Spatial Augmentation for Semi-supervised Semantic Segmentation [51.645152962504056]
  In semi-supervised semantic segmentation, data augmentation plays a crucial role in the weak-to-strong consistency regularization framework.<n>We show that spatial augmentation can contribute to model training in SSSS, despite generating inconsistent masks between the weak and strong augmentations.<n>We propose an adaptive augmentation strategy that dynamically adjusts the augmentation for each instance based on entropy.
  arXiv  Detail & Related papers  (2025-05-29T13:35:48Z)
• Message-Passing State-Space Models: Improving Graph Learning with Modern Sequence Modeling [19.10832920407789]
  We introduce a new perspective by embedding the key principles of modern SSM directly into the Message-Passing Neural Network framework.<n>Our approach, MP-SSM, enables efficient, permutation-equivariant, and long-range information propagation while preserving the architectural simplicity of message passing.
  arXiv  Detail & Related papers  (2025-05-24T14:53:07Z)
• AMMSM: Adaptive Motion Magnification and Sparse Mamba for Micro-Expression Recognition [7.084377962617903]
  We propose a multi-task learning framework named the Adaptive Motion Magnification and Sparse Mamba. This framework aims to enhance the accurate capture of micro-expressions through self-supervised subtle motion magnification. We employ evolutionary search to optimize the magnification factor and the sparsity ratios of spatial selection, followed by fine-tuning to improve performance further.
  arXiv  Detail & Related papers  (2025-03-31T13:17:43Z)
• Unsupervised Representation Learning of Complex Time Series for Maneuverability State Identification in Smart Mobility [0.0]
  In smart mobility, MTS plays a crucial role in providing temporal dynamics of behaviors such as maneuver patterns. In this work, we aim to address challenges associated with modeling MTS data collected from a vehicle using sensors. Our goal is to investigate the effectiveness of two distinct unsupervised representation learning approaches in identifying maneuvering states in smart mobility.
  arXiv  Detail & Related papers  (2024-08-26T15:16:18Z)
• Three-Stream Temporal-Shift Attention Network Based on Self-Knowledge Distillation for Micro-Expression Recognition [21.675660978188617]
  Micro-expression recognition is crucial in many fields, including criminal analysis and psychotherapy. A three-stream temporal-shift attention network based on self-knowledge distillation called SKD-TSTSAN is proposed in this paper.
  arXiv  Detail & Related papers  (2024-06-25T13:22:22Z)
• Adaptive Temporal Motion Guided Graph Convolution Network for Micro-expression Recognition [48.21696443824074]
  We propose a novel framework for micro-expression recognition, named the Adaptive Temporal Motion Guided Graph Convolution Network (ATM-GCN) Our framework excels at capturing temporal dependencies between frames across the entire clip, thereby enhancing micro-expression recognition at the clip level.
  arXiv  Detail & Related papers  (2024-06-13T10:57:24Z)
• Multi-Scale Spatio-Temporal Graph Convolutional Network for Facial Expression Spotting [11.978551396144532]
  We propose a Multi-Scale Spatio-Temporal Graph Conal Network (SpoT-CN) for facial expression spotting. We track both short- and long-term motion of facial muscles in compact sliding windows whose window length adapts to the temporal receptive field of the network. This network learns both local and global features from multiple scales of facial graph structures using our proposed facial localvolution graph pooling (FLGP)
  arXiv  Detail & Related papers  (2024-03-24T03:10:39Z)
• Unleashing Network Potentials for Semantic Scene Completion [50.95486458217653]
  This paper proposes a novel SSC framework - Adrial Modality Modulation Network (AMMNet) AMMNet introduces two core modules: a cross-modal modulation enabling the interdependence of gradient flows between modalities, and a customized adversarial training scheme leveraging dynamic gradient competition. Extensive experimental results demonstrate that AMMNet outperforms state-of-the-art SSC methods by a large margin.
  arXiv  Detail & Related papers  (2024-03-12T11:48:49Z)
• Spatio-Temporal Branching for Motion Prediction using Motion Increments [55.68088298632865]
  Human motion prediction (HMP) has emerged as a popular research topic due to its diverse applications. Traditional methods rely on hand-crafted features and machine learning techniques. We propose a noveltemporal-temporal branching network using incremental information for HMP.
  arXiv  Detail & Related papers  (2023-08-02T12:04:28Z)
• Motion Gait: Gait Recognition via Motion Excitation [5.559482051571756]
  We propose Motion Excitation Module (MEM) to guide-temporal features to focus on human parts with large dynamic changes. MEM learns the difference information between frames and intervals, so as to obtain the representation of changes temporal motion changes. We present the Fine Feature Extractor (EFF), which independently learns according to the spatial-temporal representations of human horizontal parts of individuals.
  arXiv  Detail & Related papers  (2022-06-22T13:47:14Z)
• ProgressiveMotionSeg: Mutually Reinforced Framework for Event-Based Motion Segmentation [101.19290845597918]
  This paper presents a Motion Estimation (ME) module and an Event Denoising (ED) module jointly optimized in a mutually reinforced manner. Taking temporal correlation as guidance, ED module calculates the confidence that each event belongs to real activity events, and transmits it to ME module to update energy function of motion segmentation for noise suppression.
  arXiv  Detail & Related papers  (2022-03-22T13:40:26Z)
• Short and Long Range Relation Based Spatio-Temporal Transformer for Micro-Expression Recognition [61.374467942519374]
  We propose a novel a-temporal transformer architecture -- to the best of our knowledge, the first purely transformer based approach for micro-expression recognition. The architecture comprises a spatial encoder which learns spatial patterns, a temporal dimension classification for temporal analysis, and a head. A comprehensive evaluation on three widely used spontaneous micro-expression data sets, shows that the proposed approach consistently outperforms the state of the art.
  arXiv  Detail & Related papers  (2021-12-10T22:10:31Z)
• Unsupervised Motion Representation Learning with Capsule Autoencoders [54.81628825371412]
  Motion Capsule Autoencoder (MCAE) models motion in a two-level hierarchy. MCAE is evaluated on a novel Trajectory20 motion dataset and various real-world skeleton-based human action datasets.
  arXiv  Detail & Related papers  (2021-10-01T16:52:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.