Multi-scale Dynamic and Hierarchical Relationship Modeling for Facial Action Units Recognition

• URL: http://arxiv.org/abs/2404.06443v1
• Date: Tue, 9 Apr 2024 16:45:34 GMT
• Title: Multi-scale Dynamic and Hierarchical Relationship Modeling for Facial Action Units Recognition
• Authors: Zihan Wang, Siyang Song, Cheng Luo, Songhe Deng, Weicheng Xie, Linlin Shen,
• Abstract summary: Human facial action units (AUs) are mutually related in a hierarchical manner. AUs located in the same/close facial regions show stronger relationships than those of different facial regions. This paper proposes a novel multi-scale AU model for occurrence recognition.
• Score: 38.62221940006509
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human facial action units (AUs) are mutually related in a hierarchical manner, as not only they are associated with each other in both spatial and temporal domains but also AUs located in the same/close facial regions show stronger relationships than those of different facial regions. While none of existing approach thoroughly model such hierarchical inter-dependencies among AUs, this paper proposes to comprehensively model multi-scale AU-related dynamic and hierarchical spatio-temporal relationship among AUs for their occurrences recognition. Specifically, we first propose a novel multi-scale temporal differencing network with an adaptive weighting block to explicitly capture facial dynamics across frames at different spatial scales, which specifically considers the heterogeneity of range and magnitude in different AUs' activation. Then, a two-stage strategy is introduced to hierarchically model the relationship among AUs based on their spatial distribution (i.e., local and cross-region AU relationship modelling). Experimental results achieved on BP4D and DISFA show that our approach is the new state-of-the-art in the field of AU occurrence recognition. Our code is publicly available at https://github.com/CVI-SZU/MDHR.

Related papers

• How to Bridge Spatial and Temporal Heterogeneity in Link Prediction? A Contrastive Method [11.719027225797037]
  We propose a novel textbfContrastive Learning-based textbfLink textbfPrediction model, textbfCLP. Our mymodel consistently outperforms the state-of-the-art models, demonstrating an average improvement of 10.10%, 13.44% in terms of AUC and AP.
  arXiv  Detail & Related papers  (2024-11-01T14:20:53Z)
• A Decoupled Spatio-Temporal Framework for Skeleton-based Action Segmentation [89.86345494602642]
  Existing methods are limited in weak-temporal modeling capability. We propose a Decoupled Scoupled Framework (DeST) to address the issues. DeST significantly outperforms current state-of-the-art methods with less computational complexity.
  arXiv  Detail & Related papers  (2023-12-10T09:11:39Z)
• Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition [27.34564955127377]
  The activations of Facial Action Units (AUs) mutually influence one another. Existing approaches fail to specifically and explicitly represent such cues for each pair of AUs in each facial display. This paper proposes an AU relationship modelling approach that deep learns a unique graph to explicitly describe the relationship between each pair of AUs.
  arXiv  Detail & Related papers  (2022-05-02T03:38:00Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• Modeling long-term interactions to enhance action recognition [81.09859029964323]
  We propose a new approach to under-stand actions in egocentric videos that exploits the semantics of object interactions at both frame and temporal levels. We use a region-based approach that takes as input a primary region roughly corresponding to the user hands and a set of secondary regions potentially corresponding to the interacting objects. The proposed approach outperforms the state-of-the-art in terms of action recognition on standard benchmarks.
  arXiv  Detail & Related papers  (2021-04-23T10:08:15Z)
• Multi-Level Adaptive Region of Interest and Graph Learning for Facial Action Unit Recognition [30.129452080084224]
  We propose a novel multi-level adaptive ROI and graph learning (MARGL) framework to tackle this problem. In order to incorporate the intra-level AU relation and inter-level AU regional relevance simultaneously, a multi-level AU relation graph is constructed. Experiments on BP4D and DISFA demonstrate the proposed MARGL significantly outperforms the previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-02-24T09:22:45Z)
• AU-Guided Unsupervised Domain Adaptive Facial Expression Recognition [21.126514122636966]
  This paper proposes an AU-guided unsupervised Domain Adaptive FER framework to relieve the annotation bias between different FER datasets. To achieve domain-invariant compact features, we utilize an AU-guided triplet training which randomly collects anchor-positive-negative triplets on both domains with AUs.
  arXiv  Detail & Related papers  (2020-12-18T07:17:30Z)
• Semantic Change Detection with Asymmetric Siamese Networks [71.28665116793138]
  Given two aerial images, semantic change detection aims to locate the land-cover variations and identify their change types with pixel-wise boundaries. This problem is vital in many earth vision related tasks, such as precise urban planning and natural resource management. We present an asymmetric siamese network (ASN) to locate and identify semantic changes through feature pairs obtained from modules of widely different structures.
  arXiv  Detail & Related papers  (2020-10-12T13:26:30Z)
• Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution [27.48620879003556]
  We present a new learning framework that automatically learns the latent relationships of AUs via establishing semantic correspondences between feature maps. In the heatmap regression-based network, feature maps preserve rich semantic information associated with AU intensities and locations. This motivates us to model the correlation among feature channels, which implicitly represents the co-occurrence relationship of AU intensity levels.
  arXiv  Detail & Related papers  (2020-04-20T23:55:30Z)
• Cascaded Human-Object Interaction Recognition [175.60439054047043]
  We introduce a cascade architecture for a multi-stage, coarse-to-fine HOI understanding. At each stage, an instance localization network progressively refines HOI proposals and feeds them into an interaction recognition network. With our carefully-designed human-centric relation features, these two modules work collaboratively towards effective interaction understanding.
  arXiv  Detail & Related papers  (2020-03-09T17:05:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.