Objective Class-based Micro-Expression Recognition through Simultaneous Action Unit Detection and Feature Aggregation

• URL: http://arxiv.org/abs/2012.13148v2
• Date: Tue, 23 Mar 2021 04:00:45 GMT
• Title: Objective Class-based Micro-Expression Recognition through Simultaneous Action Unit Detection and Feature Aggregation
• Authors: Ling Zhou, Qirong Mao, Ming Dong
• Abstract summary: We propose a novel deep neural network model for objective class-based Micro-Expression Recognition (MER) Our model simultaneously detects Action Units (AUs) and aggregates AU-level features into micro-expression-level representation. Our approach significantly outperforms the current state-of-the-arts in MER.
• Score: 18.35953886595087
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Micro-Expression Recognition (MER) is a challenging task as the subtle changes occur over different action regions of a face. Changes in facial action regions are formed as Action Units (AUs), and AUs in micro-expressions can be seen as the actors in cooperative group activities. In this paper, we propose a novel deep neural network model for objective class-based MER, which simultaneously detects AUs and aggregates AU-level features into micro-expression-level representation through Graph Convolutional Networks (GCN). Specifically, we propose two new strategies in our AU detection module for more effective AU feature learning: the attention mechanism and the balanced detection loss function. With those two strategies, features are learned for all the AUs in a unified model, eliminating the error-prune landmark detection process and tedious separate training for each AU. Moreover, our model incorporates a tailored objective class-based AU knowledge-graph, which facilitates the GCN to aggregate the AU-level features into a micro-expression-level feature representation. Extensive experiments on two tasks in MEGC 2018 show that our approach significantly outperforms the current state-of-the-arts in MER. Additionally, we also report our single model-based micro-expression AU detection results.

Related papers

• Facial Action Unit Detection by Adaptively Constraining Self-Attention and Causally Deconfounding Sample [53.23474626420103]
  Facial action unit (AU) detection remains a challenging task, due to the subtlety, dynamics, and diversity of AUs. We propose a novel AU detection framework called AC2D by adaptively constraining self-attention weight distribution. Our method achieves competitive performance compared to state-of-the-art AU detection approaches on challenging benchmarks.
  arXiv  Detail & Related papers  (2024-10-02T05:51:24Z)
• Local Region Perception and Relationship Learning Combined with Feature Fusion for Facial Action Unit Detection [12.677143408225167]
  We introduce our submission to the CVPR 2023 Competition on Affective Behavior Analysis in-the-wild (ABAW) We propose a single-stage trained AU detection framework. Specifically, in order to effectively extract facial local region features related to AU detection, we use a local region perception module. We also use a graph neural network-based relational learning module to capture the relationship between AUs.
  arXiv  Detail & Related papers  (2023-03-15T11:59:24Z)
• USER: Unified Semantic Enhancement with Momentum Contrast for Image-Text Retrieval [115.28586222748478]
  Image-Text Retrieval (ITR) aims at searching for the target instances that are semantically relevant to the given query from the other modality. Existing approaches typically suffer from two major limitations.
  arXiv  Detail & Related papers  (2023-01-17T12:42:58Z)
• Part-aware Prototypical Graph Network for One-shot Skeleton-based Action Recognition [57.86960990337986]
  One-shot skeleton-based action recognition poses unique challenges in learning transferable representation from base classes to novel classes. We propose a part-aware prototypical representation for one-shot skeleton-based action recognition. We demonstrate the effectiveness of our method on two public skeleton-based action recognition datasets.
  arXiv  Detail & Related papers  (2022-08-19T04:54:56Z)
• MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Units Detection [16.261362598190807]
  The Facial Action Coding System (FACS) encodes the action units (AUs) in facial images. We argue that encoding AU features just from one perspective may not capture the rich contextual information between regional and global face features. We propose a novel Multi-level Graph Reasoning Network (termed MGRR-Net) for facial AU detection.
  arXiv  Detail & Related papers  (2022-04-04T09:47:22Z)
• Weakly Supervised Regional and Temporal Learning for Facial Action Unit Recognition [36.350407471391065]
  We propose two auxiliary AU related tasks to bridge the gap between limited annotations and the model performance. A single image based optical flow estimation task is proposed to leverage the dynamic change of facial muscles. By incorporating semi-supervised learning, we propose an end-to-end trainable framework named weakly supervised regional and temporal learning.
  arXiv  Detail & Related papers  (2022-04-01T12:02:01Z)
• Meta Auxiliary Learning for Facial Action Unit Detection [84.22521265124806]
  We consider learning AU detection and facial expression recognition in a multi-task manner. The performance of the AU detection task cannot be always enhanced due to the negative transfer in the multi-task scenario. We propose a Meta Learning method (MAL) that automatically selects highly related FE samples by learning adaptative weights for the training FE samples in a meta learning manner.
  arXiv  Detail & Related papers  (2021-05-14T02:28:40Z)
• MER-GCN: Micro Expression Recognition Based on Relation Modeling with Graph Convolutional Network [24.69269586706002]
  We propose an end-to-end AU-oriented graph classification network, namely MER-GCN, which uses 3D ConvNets to extract AU features and applies GCN layers to discover the dependency laying between AU nodes for ME categorization. To our best knowledge, this work is the first end-to-end architecture for Micro-Expression Recognition (MER) using AUs based GCN.
  arXiv  Detail & Related papers  (2020-04-19T17:25:30Z)
• J$\hat{\text{A}}$A-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attention [57.51255553918323]
  We propose a novel end-to-end deep learning framework for joint AU detection and face alignment. Our framework significantly outperforms the state-of-the-art AU detection methods on the challenging BP4D, DISFA, GFT and BP4D+ benchmarks.
  arXiv  Detail & Related papers  (2020-03-18T12:50:19Z)
• Global Context-Aware Progressive Aggregation Network for Salient Object Detection [117.943116761278]
  We propose a novel network named GCPANet to integrate low-level appearance features, high-level semantic features, and global context features. We show that the proposed approach outperforms the state-of-the-art methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2020-03-02T04:26:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.