Trend-Aware Supervision: On Learning Invariance for Semi-Supervised Facial Action Unit Intensity Estimation

• URL: http://arxiv.org/abs/2503.08078v1
• Date: Tue, 11 Mar 2025 06:21:09 GMT
• Title: Trend-Aware Supervision: On Learning Invariance for Semi-Supervised Facial Action Unit Intensity Estimation
• Authors: Yingjie Chen, Jiarui Zhang, Tao Wang, Yun Liang,
• Abstract summary: We propose textbfTrend-textbfAware textbfSupervision (TAS), which pursues three kinds of trend awareness, including intra-trend ranking awareness, intra-trend speed awareness, and inter-trend subject awareness.<n>TAS alleviates the spurious correlation problem by raising trend awareness during training to learn AU-specific features that represent the corresponding facial appearance changes.
• Score: 9.801711534323605
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the increasing need for facial behavior analysis, semi-supervised AU intensity estimation using only keyframe annotations has emerged as a practical and effective solution to relieve the burden of annotation. However, the lack of annotations makes the spurious correlation problem caused by AU co-occurrences and subject variation much more prominent, leading to non-robust intensity estimation that is entangled among AUs and biased among subjects. We observe that trend information inherent in keyframe annotations could act as extra supervision and raising the awareness of AU-specific facial appearance changing trends during training is the key to learning invariant AU-specific features. To this end, we propose \textbf{T}rend-\textbf{A}ware \textbf{S}upervision (TAS), which pursues three kinds of trend awareness, including intra-trend ranking awareness, intra-trend speed awareness, and inter-trend subject awareness. TAS alleviates the spurious correlation problem by raising trend awareness during training to learn AU-specific features that represent the corresponding facial appearance changes, to achieve intensity estimation invariance. Experiments conducted on two commonly used AU benchmark datasets, BP4D and DISFA, show the effectiveness of each kind of awareness. And under trend-aware supervision, the performance can be improved without extra computational or storage costs during inference.

Related papers

• Latent Adversarial Training Improves the Representation of Refusal [41.94295877935867]
  We show how Latent Adversarial Training (LAT) reorganizes the refusal behavior in the model's latent space. LAT significantly alters the refusal representation, concentrating it in the first two SVD components. Our findings suggest that LAT's training perturbations enable a more comprehensive representation of refusal behavior.
  arXiv  Detail & Related papers  (2025-04-26T09:40:31Z)
• Interpretable Vital Sign Forecasting with Model Agnostic Attention Maps [5.354055742467353]
  This paper introduces a framework that combines a deep learning model with an attention mechanism that highlights the critical time steps in the forecasting process. We show that the attention mechanism could be adapted to various black box time series forecasting models such as N-HiTS and N-BEATS.
  arXiv  Detail & Related papers  (2024-05-02T20:19:07Z)
• Augmenting Unsupervised Reinforcement Learning with Self-Reference [63.68018737038331]
  Humans possess the ability to draw on past experiences explicitly when learning new tasks. We propose the Self-Reference (SR) approach, an add-on module explicitly designed to leverage historical information. Our approach achieves state-of-the-art results in terms of Interquartile Mean (IQM) performance and Optimality Gap reduction on the Unsupervised Reinforcement Learning Benchmark.
  arXiv  Detail & Related papers  (2023-11-16T09:07:34Z)
• Implicit Counterfactual Data Augmentation for Robust Learning [24.795542869249154]
  This study proposes an Implicit Counterfactual Data Augmentation method to remove spurious correlations and make stable predictions. Experiments have been conducted across various biased learning scenarios covering both image and text datasets.
  arXiv  Detail & Related papers  (2023-04-26T10:36:40Z)
• More Than Just Attention: Learning Cross-Modal Attentions with Contrastive Constraints [63.08768589044052]
  We propose Contrastive Content Re-sourcing ( CCR) and Contrastive Content Swapping ( CCS) constraints to address such limitation. CCR and CCS constraints supervise the training of attention models in a contrastive learning manner without requiring explicit attention annotations. Experiments on both Flickr30k and MS-COCO datasets demonstrate that integrating these attention constraints into two state-of-the-art attention-based models improves the model performance.
  arXiv  Detail & Related papers  (2021-05-20T08:48:10Z)
• Robust Pre-Training by Adversarial Contrastive Learning [120.33706897927391]
  Recent work has shown that, when integrated with adversarial training, self-supervised pre-training can lead to state-of-the-art robustness. We improve robustness-aware self-supervised pre-training by learning representations consistent under both data augmentations and adversarial perturbations.
  arXiv  Detail & Related papers  (2020-10-26T04:44:43Z)
• Controlling the Interaction Between Generation and Inference in Semi-Supervised Variational Autoencoders Using Importance Weighting [0.9582466286528458]
  Variational Autoencoders (VAEs) are widely used for semi-supervised learning. We observe that they use the posterior of the learned generative model to guide the inference model in learning the partially observed latent variable. Using importance weighting, we derive two novel objectives that prioritize either one of the partially observed latent variable, or the unobserved latent variable.
  arXiv  Detail & Related papers  (2020-10-13T17:01:40Z)
• Accurate and Robust Feature Importance Estimation under Distribution Shifts [49.58991359544005]
  PRoFILE is a novel feature importance estimation method. We show significant improvements over state-of-the-art approaches, both in terms of fidelity and robustness.
  arXiv  Detail & Related papers  (2020-09-30T05:29:01Z)
• Attention Meets Perturbations: Robust and Interpretable Attention with Adversarial Training [7.106986689736828]
  We propose a general training technique for natural language processing tasks, including AT for attention (Attention AT) and more interpretable AT for attention (Attention iAT) The proposed techniques improved the prediction performance and the model interpretability by exploiting the mechanisms with AT.
  arXiv  Detail & Related papers  (2020-09-25T07:26:45Z)
• Spectrum-Guided Adversarial Disparity Learning [52.293230153385124]
  We propose a novel end-to-end knowledge directed adversarial learning framework. It portrays the class-conditioned intraclass disparity using two competitive encoding distributions and learns the purified latent codes by denoising learned disparity. The experiments on four HAR benchmark datasets demonstrate the robustness and generalization of our proposed methods over a set of state-of-the-art.
  arXiv  Detail & Related papers  (2020-07-14T05:46:27Z)
• Cost-effective Interactive Attention Learning with Neural Attention Processes [79.8115563067513]
  We propose a novel interactive learning framework which we refer to as Interactive Attention Learning (IAL) IAL is prone to overfitting due to scarcity of human annotations, and requires costly retraining. We tackle these challenges by proposing a sample-efficient attention mechanism and a cost-effective reranking algorithm for instances and features.
  arXiv  Detail & Related papers  (2020-06-09T17:36:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.