Learning Contrastive Feature Representations for Facial Action Unit Detection

• URL: http://arxiv.org/abs/2402.06165v5
• Date: Thu, 17 Oct 2024 14:10:16 GMT
• Title: Learning Contrastive Feature Representations for Facial Action Unit Detection
• Authors: Ziqiao Shang, Bin Liu, Fengmao Lv, Fei Teng, Tianrui Li,
• Abstract summary: Facial action unit (AU) detection has long encountered the challenge of detecting subtle feature differences when AUs activate. We introduce a novel contrastive learning framework aimed for AU detection that incorporates both self-supervised and supervised signals.
• Score: 13.834540490373818
• License:
• Abstract: Facial action unit (AU) detection has long encountered the challenge of detecting subtle feature differences when AUs activate. Existing methods often rely on encoding pixel-level information of AUs, which not only encodes additional redundant information but also leads to increased model complexity and limited generalizability. Additionally, the accuracy of AU detection is negatively impacted by the class imbalance issue of each AU type, and the presence of noisy and false AU labels. In this paper, we introduce a novel contrastive learning framework aimed for AU detection that incorporates both self-supervised and supervised signals, thereby enhancing the learning of discriminative features for accurate AU detection. To tackle the class imbalance issue, we employ a negative sample re-weighting strategy that adjusts the step size of updating parameters for minority and majority class samples. Moreover, to address the challenges posed by noisy and false AU labels, we employ a sampling technique that encompasses three distinct types of positive sample pairs. This enables us to inject self-supervised signals into the supervised signal, effectively mitigating the adverse effects of noisy labels. Our experimental assessments, conducted on four widely-utilized benchmark datasets (BP4D, DISFA, GFT and Aff-Wild2), underscore the superior performance of our approach compared to state-of-the-art methods of AU detection. Our code is available at \url{https://github.com/Ziqiao-Shang/AUNCE}.

Related papers

• Facial Action Unit Detection by Adaptively Constraining Self-Attention and Causally Deconfounding Sample [53.23474626420103]
  Facial action unit (AU) detection remains a challenging task, due to the subtlety, dynamics, and diversity of AUs. We propose a novel AU detection framework called AC2D by adaptively constraining self-attention weight distribution. Our method achieves competitive performance compared to state-of-the-art AU detection approaches on challenging benchmarks.
  arXiv  Detail & Related papers  (2024-10-02T05:51:24Z)
• Robust Tiny Object Detection in Aerial Images amidst Label Noise [50.257696872021164]
  This study addresses the issue of tiny object detection under noisy label supervision. We propose a DeNoising Tiny Object Detector (DN-TOD), which incorporates a Class-aware Label Correction scheme. Our method can be seamlessly integrated into both one-stage and two-stage object detection pipelines.
  arXiv  Detail & Related papers  (2024-01-16T02:14:33Z)
• CL-Flow:Strengthening the Normalizing Flows by Contrastive Learning for Better Anomaly Detection [1.951082473090397]
  We propose a self-supervised anomaly detection approach that combines contrastive learning with 2D-Flow. Compared to mainstream unsupervised approaches, our self-supervised method demonstrates superior detection accuracy, fewer additional model parameters, and faster inference speed. Our approach showcases new state-of-the-art results, achieving a performance of 99.6% in image-level AUROC on the MVTecAD dataset and 96.8% in image-level AUROC on the BTAD dataset.
  arXiv  Detail & Related papers  (2023-11-12T10:07:03Z)
• Augment and Criticize: Exploring Informative Samples for Semi-Supervised Monocular 3D Object Detection [64.65563422852568]
  We improve the challenging monocular 3D object detection problem with a general semi-supervised framework. We introduce a novel, simple, yet effective Augment and Criticize' framework that explores abundant informative samples from unlabeled data. The two new detectors, dubbed 3DSeMo_DLE and 3DSeMo_FLEX, achieve state-of-the-art results with remarkable improvements for over 3.5% AP_3D/BEV (Easy) on KITTI.
  arXiv  Detail & Related papers  (2023-03-20T16:28:15Z)
• PASS: Peer-Agreement based Sample Selection for training with Noisy Labels [16.283722126438125]
  The prevalence of noisy-label samples poses a significant challenge in deep learning, inducing overfitting effects. Current methodologies often rely on the small-loss hypothesis or feature-based selection to separate noisy- and clean-label samples. We propose a new noisy-label detection method, termed Peer-Agreement based Sample Selection (PASS), to address this problem.
  arXiv  Detail & Related papers  (2023-03-20T00:35:33Z)
• SIOD: Single Instance Annotated Per Category Per Image for Object Detection [67.64774488115299]
  We propose the Single Instance annotated Object Detection (SIOD), requiring only one instance annotation for each existing category in an image. Degraded from inter-task (WSOD) or inter-image (SSOD) discrepancies to the intra-image discrepancy, SIOD provides more reliable and rich prior knowledge for mining the rest of unlabeled instances. Under the SIOD setting, we propose a simple yet effective framework, termed Dual-Mining (DMiner), which consists of a Similarity-based Pseudo Label Generating module (SPLG) and a Pixel-level Group Contrastive Learning module (PGCL)
  arXiv  Detail & Related papers  (2022-03-29T08:49:51Z)
• Learning to Detect Instance-level Salient Objects Using Complementary Image Labels [55.049347205603304]
  We present the first weakly-supervised approach to the salient instance detection problem. We propose a novel weakly-supervised network with three branches: a Saliency Detection Branch leveraging class consistency information to locate candidate objects; a Boundary Detection Branch exploiting class discrepancy information to delineate object boundaries; and a Centroid Detection Branch using subitizing information to detect salient instance centroids.
  arXiv  Detail & Related papers  (2021-11-19T10:15:22Z)
• Meta Auxiliary Learning for Facial Action Unit Detection [84.22521265124806]
  We consider learning AU detection and facial expression recognition in a multi-task manner. The performance of the AU detection task cannot be always enhanced due to the negative transfer in the multi-task scenario. We propose a Meta Learning method (MAL) that automatically selects highly related FE samples by learning adaptative weights for the training FE samples in a meta learning manner.
  arXiv  Detail & Related papers  (2021-05-14T02:28:40Z)
• Multi-Objective Interpolation Training for Robustness to Label Noise [17.264550056296915]
  We show that standard supervised contrastive learning degrades in the presence of label noise. We propose a novel label noise detection method that exploits the robust feature representations learned via contrastive learning. Experiments on synthetic and real-world noise benchmarks demonstrate that MOIT/MOIT+ achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2020-12-08T15:01:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.