AUFormer: Vision Transformers are Parameter-Efficient Facial Action Unit Detectors

• URL: http://arxiv.org/abs/2403.04697v2
• Date: Tue, 9 Jul 2024 15:15:21 GMT
• Title: AUFormer: Vision Transformers are Parameter-Efficient Facial Action Unit Detectors
• Authors: Kaishen Yuan, Zitong Yu, Xin Liu, Weicheng Xie, Huanjing Yue, Jingyu Yang,
• Abstract summary: Existing methods suffer from overfitting issues due to the utilization of a large number of learnable parameters. PETL provides a promising paradigm to address these challenges. We propose a novel Mixture-of-Knowledge Expert (MoKE) collaboration mechanism.
• Score: 31.547624650827395
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Facial Action Units (AU) is a vital concept in the realm of affective computing, and AU detection has always been a hot research topic. Existing methods suffer from overfitting issues due to the utilization of a large number of learnable parameters on scarce AU-annotated datasets or heavy reliance on substantial additional relevant data. Parameter-Efficient Transfer Learning (PETL) provides a promising paradigm to address these challenges, whereas its existing methods lack design for AU characteristics. Therefore, we innovatively investigate PETL paradigm to AU detection, introducing AUFormer and proposing a novel Mixture-of-Knowledge Expert (MoKE) collaboration mechanism. An individual MoKE specific to a certain AU with minimal learnable parameters first integrates personalized multi-scale and correlation knowledge. Then the MoKE collaborates with other MoKEs in the expert group to obtain aggregated information and inject it into the frozen Vision Transformer (ViT) to achieve parameter-efficient AU detection. Additionally, we design a Margin-truncated Difficulty-aware Weighted Asymmetric Loss (MDWA-Loss), which can encourage the model to focus more on activated AUs, differentiate the difficulty of unactivated AUs, and discard potential mislabeled samples. Extensive experiments from various perspectives, including within-domain, cross-domain, data efficiency, and micro-expression domain, demonstrate AUFormer's state-of-the-art performance and robust generalization abilities without relying on additional relevant data. The code for AUFormer is available at https://github.com/yuankaishen2001/AUFormer.

Related papers

• Facial Action Unit Detection by Adaptively Constraining Self-Attention and Causally Deconfounding Sample [53.23474626420103]
  Facial action unit (AU) detection remains a challenging task, due to the subtlety, dynamics, and diversity of AUs. We propose a novel AU detection framework called AC2D by adaptively constraining self-attention weight distribution. Our method achieves competitive performance compared to state-of-the-art AU detection approaches on challenging benchmarks.
  arXiv  Detail & Related papers  (2024-10-02T05:51:24Z)
• Feature Attenuation of Defective Representation Can Resolve Incomplete Masking on Anomaly Detection [1.0358639819750703]
  In unsupervised anomaly detection (UAD) research, it is necessary to develop a computationally efficient and scalable solution. We revisit the reconstruction-by-inpainting approach and rethink to improve it by analyzing strengths and weaknesses. We propose Feature Attenuation of Defective Representation (FADeR) that only employs two layers which attenuates feature information of anomaly reconstruction.
  arXiv  Detail & Related papers  (2024-07-05T15:44:53Z)
• Learning Contrastive Feature Representations for Facial Action Unit Detection [13.834540490373818]
  Facial action unit (AU) detection has long encountered the challenge of detecting subtle feature differences when AUs activate. We introduce a novel contrastive learning framework aimed for AU detection that incorporates both self-supervised and supervised signals.
  arXiv  Detail & Related papers  (2024-02-09T03:48:20Z)
• Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective [106.92016199403042]
  We empirically investigate knowledge transfer from larger to smaller models through a parametric perspective. We employ sensitivity-based techniques to extract and align knowledge-specific parameters between different large language models. Our findings highlight the critical factors contributing to the process of parametric knowledge transfer.
  arXiv  Detail & Related papers  (2023-10-17T17:58:34Z)
• Physics Inspired Hybrid Attention for SAR Target Recognition [61.01086031364307]
  We propose a physics inspired hybrid attention (PIHA) mechanism and the once-for-all (OFA) evaluation protocol to address the issues. PIHA leverages the high-level semantics of physical information to activate and guide the feature group aware of local semantics of target. Our method outperforms other state-of-the-art approaches in 12 test scenarios with same ASC parameters.
  arXiv  Detail & Related papers  (2023-09-27T14:39:41Z)
• Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection [37.841035367349434]
  Facial Action Unit (AU) detection is a crucial task in affective computing and social robotics. Previous methods used fixed AU correlations based on expert experience or statistical rules on specific benchmarks. This paper proposes a novel self-adjusting AU-correlation learning (SACL) method with less proposes for AU detection.
  arXiv  Detail & Related papers  (2023-08-15T13:43:48Z)
• FAN-Trans: Online Knowledge Distillation for Facial Action Unit Detection [45.688712067285536]
  Leveraging the online knowledge distillation framework, we propose the FANTrans" method for AU detection. Our model consists of a hybrid network of convolution and transformer blocks to learn per-AU features and to model AU co-occurrences.
  arXiv  Detail & Related papers  (2022-11-11T11:35:33Z)
• Discover, Explanation, Improvement: An Automatic Slice Detection Framework for Natural Language Processing [72.14557106085284]
  slice detection models (SDM) automatically identify underperforming groups of datapoints. This paper proposes a benchmark named "Discover, Explain, improve (DEIM)" for classification NLP tasks. Our evaluation shows that Edisa can accurately select error-prone datapoints with informative semantic features.
  arXiv  Detail & Related papers  (2022-11-08T19:00:00Z)
• Meta Auxiliary Learning for Facial Action Unit Detection [84.22521265124806]
  We consider learning AU detection and facial expression recognition in a multi-task manner. The performance of the AU detection task cannot be always enhanced due to the negative transfer in the multi-task scenario. We propose a Meta Learning method (MAL) that automatically selects highly related FE samples by learning adaptative weights for the training FE samples in a meta learning manner.
  arXiv  Detail & Related papers  (2021-05-14T02:28:40Z)
• Active Feature Acquisition with Generative Surrogate Models [11.655069211977464]
  In this work, we consider models that perform active feature acquisition (AFA) and query the environment for unobserved features. Our work reformulates the Markov decision process (MDP) that underlies the AFA problem as a generative modeling task. We propose learning a generative surrogate model ( GSM) that captures the dependencies among input features to assess potential information gain from acquisitions.
  arXiv  Detail & Related papers  (2020-10-06T02:10:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.