From Macro to Micro: Boosting micro-expression recognition via pre-training on macro-expression videos

• URL: http://arxiv.org/abs/2405.16451v2
• Date: Tue, 4 Jun 2024 05:15:50 GMT
• Title: From Macro to Micro: Boosting micro-expression recognition via pre-training on macro-expression videos
• Authors: Hanting Li, Hongjing Niu, Feng Zhao,
• Abstract summary: Micro-expression recognition (MER) has drawn increasing attention in recent years due to its potential applications in intelligent medical and lie detection. We propose a generalized transfer learning paradigm, called textbfMAcro-expression textbfTO textbfMIcro-expression (MA2MI) Under our paradigm, networks can learns the ability to represent subtle facial movement by reconstructing future frames.
• Score: 9.472210792839023
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Micro-expression recognition (MER) has drawn increasing attention in recent years due to its potential applications in intelligent medical and lie detection. However, the shortage of annotated data has been the major obstacle to further improve deep-learning based MER methods. Intuitively, utilizing sufficient macro-expression data to promote MER performance seems to be a feasible solution. However, the facial patterns of macro-expressions and micro-expressions are significantly different, which makes naive transfer learning methods difficult to deploy directly. To tacle this issue, we propose a generalized transfer learning paradigm, called \textbf{MA}cro-expression \textbf{TO} \textbf{MI}cro-expression (MA2MI). Under our paradigm, networks can learns the ability to represent subtle facial movement by reconstructing future frames. In addition, we also propose a two-branch micro-action network (MIACNet) to decouple facial position features and facial action features, which can help the network more accurately locate facial action locations. Extensive experiments on three popular MER benchmarks demonstrate the superiority of our method.

Related papers

• Three-Stream Temporal-Shift Attention Network Based on Self-Knowledge Distillation for Micro-Expression Recognition [21.675660978188617]
  Micro-expression recognition is crucial in many fields, including criminal analysis and psychotherapy. A three-stream temporal-shift attention network based on self-knowledge distillation called SKD-TSTSAN is proposed in this paper.
  arXiv  Detail & Related papers  (2024-06-25T13:22:22Z)
• Adaptive Temporal Motion Guided Graph Convolution Network for Micro-expression Recognition [48.21696443824074]
  We propose a novel framework for micro-expression recognition, named the Adaptive Temporal Motion Guided Graph Convolution Network (ATM-GCN) Our framework excels at capturing temporal dependencies between frames across the entire clip, thereby enhancing micro-expression recognition at the clip level.
  arXiv  Detail & Related papers  (2024-06-13T10:57:24Z)
• DualFocus: Integrating Macro and Micro Perspectives in Multi-modal Large Language Models [85.4852517178828]
  We present DualFocus, a framework for integrating macro and micro perspectives within multi-modal large language models (MLLMs) We demonstrate DualFocus's superiority in balancing detailed examination with holistic insight, significantly reducing hallucination instances in MLLMs.
  arXiv  Detail & Related papers  (2024-02-22T18:26:02Z)
• AU-aware graph convolutional network for Macro- and Micro-expression spotting [44.507747407072685]
  We propose a graph convolutional-based network, called Action-Unit-aWare Graph Convolutional Network (AUW-GCN) To inject prior information and to cope with the problem of small datasets, AU-related statistics are encoded into the network. Our results outperform baseline methods consistently and achieve new SOTA performance in two benchmark datasets.
  arXiv  Detail & Related papers  (2023-03-16T07:00:36Z)
• Transferring Dual Stochastic Graph Convolutional Network for Facial Micro-expression Recognition [7.62031665958404]
  This paper presents a transferring dual Graph Convolutional Network (GCN) model. We propose a graph construction method and dual graph convolutional network to extract more discriminative features from the micro-expression images. Our proposed method achieves state-of-the-art performance on recently released MMEW benchmarks.
  arXiv  Detail & Related papers  (2022-03-10T07:41:18Z)
• Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms [52.58031087639394]
  micro-expressions are involuntary and transient facial expressions. They can provide important information in a broad range of applications such as lie detection, criminal detection, etc. Since micro-expressions are transient and of low intensity, their detection and recognition is difficult and relies heavily on expert experiences.
  arXiv  Detail & Related papers  (2022-01-30T05:14:13Z)
• MMNet: Muscle motion-guided network for micro-expression recognition [2.032432845751978]
  We propose a robust micro-expression recognition framework, namely muscle motion-guided network (MMNet) Specifically, a continuous attention (CA) block is introduced to focus on modeling local subtle muscle motion patterns with little identity information. Our approach outperforms state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2022-01-14T04:05:49Z)
• Short and Long Range Relation Based Spatio-Temporal Transformer for Micro-Expression Recognition [61.374467942519374]
  We propose a novel a-temporal transformer architecture -- to the best of our knowledge, the first purely transformer based approach for micro-expression recognition. The architecture comprises a spatial encoder which learns spatial patterns, a temporal dimension classification for temporal analysis, and a head. A comprehensive evaluation on three widely used spontaneous micro-expression data sets, shows that the proposed approach consistently outperforms the state of the art.
  arXiv  Detail & Related papers  (2021-12-10T22:10:31Z)
• Shallow Optical Flow Three-Stream CNN for Macro- and Micro-Expression Spotting from Long Videos [15.322908569777551]
  We propose a model to predict a score that captures the likelihood of a frame being in an expression interval. We demonstrate the efficacy and efficiency of the proposed approach on the recent MEGC 2020 benchmark.
  arXiv  Detail & Related papers  (2021-06-11T16:19:48Z)
• An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge [24.495792982803124]
  Facial micro-expressions indicate brief and subtle facial movements that appear during emotional communication. In recent years, micro-expression recognition (MER) has drawn much attention because it can benefit a wide range of applications. MER approaches from three novel aspects: macro-to-micro adaptation, recognition based on key apex frames, and recognition based on facial action units.
  arXiv  Detail & Related papers  (2020-12-21T13:20:17Z)
• Micro-expression spotting: A new benchmark [74.69928316848866]
  Micro-expressions (MEs) are brief and involuntary facial expressions that occur when people are trying to hide their true feelings or conceal their emotions. In the computer vision field, the study of MEs can be divided into two main tasks, spotting and recognition. This paper introduces an extension of the SMIC-E database, namely the SMIC-E-Long database, which is a new challenging benchmark for ME spotting.
  arXiv  Detail & Related papers  (2020-07-24T09:18:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.