Attention Based Relation Network for Facial Action Units Recognition

• URL: http://arxiv.org/abs/2210.13988v1
• Date: Sun, 23 Oct 2022 11:26:53 GMT
• Title: Attention Based Relation Network for Facial Action Units Recognition
• Authors: Yao Wei and Haoxiang Wang and Mingze Sun and Jiawang Liu
• Abstract summary: We propose a novel Attention Based Relation Network (ABRNet) for AU recognition. ABRNet uses several relation learning layers to automatically capture different AU relations. Our approach achieves state-of-the-art performance on the DISFA and DISFA+ datasets.
• Score: 8.522262699196412
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Facial action unit (AU) recognition is essential to facial expression analysis. Since there are highly positive or negative correlations between AUs, some existing AU recognition works have focused on modeling AU relations. However, previous relationship-based approaches typically embed predefined rules into their models and ignore the impact of various AU relations in different crowds. In this paper, we propose a novel Attention Based Relation Network (ABRNet) for AU recognition, which can automatically capture AU relations without unnecessary or even disturbing predefined rules. ABRNet uses several relation learning layers to automatically capture different AU relations. The learned AU relation features are then fed into a self-attention fusion module, which aims to refine individual AU features with attention weights to enhance the feature robustness. Furthermore, we propose an AU relation dropout strategy and AU relation loss (AUR-Loss) to better model AU relations, which can further improve AU recognition. Extensive experiments show that our approach achieves state-of-the-art performance on the DISFA and DISFA+ datasets.

Related papers

• Facial Action Unit Detection by Adaptively Constraining Self-Attention and Causally Deconfounding Sample [53.23474626420103]
  Facial action unit (AU) detection remains a challenging task, due to the subtlety, dynamics, and diversity of AUs. We propose a novel AU detection framework called AC2D by adaptively constraining self-attention weight distribution. Our method achieves competitive performance compared to state-of-the-art AU detection approaches on challenging benchmarks.
  arXiv  Detail & Related papers  (2024-10-02T05:51:24Z)
• Towards End-to-End Explainable Facial Action Unit Recognition via Vision-Language Joint Learning [48.70249675019288]
  We propose an end-to-end Vision-Language joint learning network for explainable facial action units (AUs) recognition. The proposed approach achieves superior performance over the state-of-the-art methods on most metrics.
  arXiv  Detail & Related papers  (2024-08-01T15:35:44Z)
• Contrastive Learning of Person-independent Representations for Facial Action Unit Detection [70.60587475492065]
  We formulate the self-supervised AU representation learning signals in two-fold. We contrast learn the AU representation within a video clip and devise a cross-identity reconstruction mechanism to learn the person-independent representations. Our method outperforms other contrastive learning methods and significantly closes the performance gap between the self-supervised and supervised AU detection approaches.
  arXiv  Detail & Related papers  (2024-03-06T01:49:28Z)
• Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection [37.841035367349434]
  Facial Action Unit (AU) detection is a crucial task in affective computing and social robotics. Previous methods used fixed AU correlations based on expert experience or statistical rules on specific benchmarks. This paper proposes a novel self-adjusting AU-correlation learning (SACL) method with less proposes for AU detection.
  arXiv  Detail & Related papers  (2023-08-15T13:43:48Z)
• Local Region Perception and Relationship Learning Combined with Feature Fusion for Facial Action Unit Detection [12.677143408225167]
  We introduce our submission to the CVPR 2023 Competition on Affective Behavior Analysis in-the-wild (ABAW) We propose a single-stage trained AU detection framework. Specifically, in order to effectively extract facial local region features related to AU detection, we use a local region perception module. We also use a graph neural network-based relational learning module to capture the relationship between AUs.
  arXiv  Detail & Related papers  (2023-03-15T11:59:24Z)
• Self-supervised Facial Action Unit Detection with Region and Relation Learning [5.182661263082065]
  We propose a novel self-supervised framework for AU detection with the region and relation learning. An improved Optimal Transport (OT) algorithm is introduced to exploit the correlation characteristics among AUs. Swin Transformer is exploited to model the long-distance dependencies within each AU region during feature learning.
  arXiv  Detail & Related papers  (2023-03-10T05:22:45Z)
• Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition [27.34564955127377]
  The activations of Facial Action Units (AUs) mutually influence one another. Existing approaches fail to specifically and explicitly represent such cues for each pair of AUs in each facial display. This paper proposes an AU relationship modelling approach that deep learns a unique graph to explicitly describe the relationship between each pair of AUs.
  arXiv  Detail & Related papers  (2022-05-02T03:38:00Z)
• AU-Expression Knowledge Constrained Representation Learning for Facial Expression Recognition [79.8779790682205]
  We propose an AU-Expression Knowledge Constrained Representation Learning (AUE-CRL) framework to learn the AU representations without AU annotations and adaptively use representations to facilitate facial expression recognition. We conduct experiments on the challenging uncontrolled datasets to demonstrate the superiority of the proposed framework over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-12-29T03:42:04Z)
• Learning to Decouple Relations: Few-Shot Relation Classification with Entity-Guided Attention and Confusion-Aware Training [49.9995628166064]
  We propose CTEG, a model equipped with two mechanisms to learn to decouple easily-confused relations. On the one hand, an EGA mechanism is introduced to guide the attention to filter out information causing confusion. On the other hand, a Confusion-Aware Training (CAT) method is proposed to explicitly learn to distinguish relations.
  arXiv  Detail & Related papers  (2020-10-21T11:07:53Z)
• Actor-Context-Actor Relation Network for Spatio-Temporal Action Localization [47.61419011906561]
  ACAR-Net builds upon a novel High-order Relation Reasoning Operator to enable indirect reasoning fortemporal action localization. Our method ranks first in the AVA-Kineticsaction localization task of ActivityNet Challenge 2020.
  arXiv  Detail & Related papers  (2020-06-14T18:51:49Z)
• Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution [27.48620879003556]
  We present a new learning framework that automatically learns the latent relationships of AUs via establishing semantic correspondences between feature maps. In the heatmap regression-based network, feature maps preserve rich semantic information associated with AU intensities and locations. This motivates us to model the correlation among feature channels, which implicitly represents the co-occurrence relationship of AU intensity levels.
  arXiv  Detail & Related papers  (2020-04-20T23:55:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.