V-NAW: Video-based Noise-aware Adaptive Weighting for Facial Expression Recognition

• URL: http://arxiv.org/abs/2503.15970v1
• Date: Thu, 20 Mar 2025 09:13:34 GMT
• Title: V-NAW: Video-based Noise-aware Adaptive Weighting for Facial Expression Recognition
• Authors: JunGyu Lee, Kunyoung Lee, Haesol Park, Ig-Jae Kim, Gi Pyo Nam,
• Abstract summary: 8th Affective Behavior Analysis in-the-Wild (ABAW) Challenge aims to assess human emotions using the video-based Aff-Wild2 dataset.<n>This challenge includes various tasks, including the video-based EXPR recognition track, which is our primary focus.<n>We propose Video-based Noise-aware Adaptive Weighting (V-NAW), which adaptively assigns importance to each frame in a clip to address label ambiguity and effectively capture temporal variations in facial expressions.
• Score: 9.57248169951292
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Facial Expression Recognition (FER) plays a crucial role in human affective analysis and has been widely applied in computer vision tasks such as human-computer interaction and psychological assessment. The 8th Affective Behavior Analysis in-the-Wild (ABAW) Challenge aims to assess human emotions using the video-based Aff-Wild2 dataset. This challenge includes various tasks, including the video-based EXPR recognition track, which is our primary focus. In this paper, we demonstrate that addressing label ambiguity and class imbalance, which are known to cause performance degradation, can lead to meaningful performance improvements. Specifically, we propose Video-based Noise-aware Adaptive Weighting (V-NAW), which adaptively assigns importance to each frame in a clip to address label ambiguity and effectively capture temporal variations in facial expressions. Furthermore, we introduce a simple and effective augmentation strategy to reduce redundancy between consecutive frames, which is a primary cause of overfitting. Through extensive experiments, we validate the effectiveness of our approach, demonstrating significant improvements in video-based FER performance.

Related papers

• HSEmotion Team at ABAW-8 Competition: Audiovisual Ambivalence/Hesitancy, Emotional Mimicry Intensity and Facial Expression Recognition [16.860963320038902]
  This article presents our results for the eighth Affective Behavior Analysis in-the-Wild (ABAW) competition.<n>We combine facial emotional descriptors extracted by pre-trained models with acoustic features and embeddings of texts recognized from speech.<n>The video-level prediction of emotional mimicry intensity is implemented by simply aggregating frame-level features and training a multi-layered perceptron.
  arXiv  Detail & Related papers  (2025-03-13T14:21:46Z)
• When Pre-trained Visual Representations Fall Short: Limitations in Visuo-Motor Robot Learning [19.394253159981577]
  integration of pre-trained visual representations into visuo-motor robot learning has emerged as a promising alternative to training visual encoders from scratch. PVRs face critical challenges in the context of policy learning, including temporal entanglement and an inability to generalise even in the presence of minor scene perturbations. This work identifies these shortcomings and proposes solutions to address them. First, we augment PVR features with temporal perception and a sense of task completion, effectively disangling them in time. Second, we introduce a module that learns to selectively attend to task-relevant local features, enhancing robustness when evaluated on out-of-distribution
  arXiv  Detail & Related papers  (2025-02-05T15:25:46Z)
• SVFAP: Self-supervised Video Facial Affect Perceiver [42.16505961654868]
  Motivated by the recent success of self-supervised learning in computer vision, this paper introduces a self-supervised approach, termed Self-supervised Video Facial Affect Perceiver (SVFAP) To address the dilemma faced by supervised methods, SVFAP leverages masked video autoencoding to perform self-supervised pre-training on massive unlabeled facial videos. To verify the effectiveness of our method, we conduct experiments on nine datasets spanning three downstream tasks, including dynamic facial expression recognition, dimensional emotion recognition, and personality recognition.
  arXiv  Detail & Related papers  (2023-12-31T07:44:05Z)
• From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos [88.08209394979178]
  Dynamic facial expression recognition (DFER) in the wild is still hindered by data limitations. We introduce a novel Static-to-Dynamic model (S2D) that leverages existing SFER knowledge and dynamic information implicitly encoded in extracted facial landmark-aware features.
  arXiv  Detail & Related papers  (2023-12-09T03:16:09Z)
• SS-VAERR: Self-Supervised Apparent Emotional Reaction Recognition from Video [61.21388780334379]
  This work focuses on the apparent emotional reaction recognition from the video-only input, conducted in a self-supervised fashion. The network is first pre-trained on different self-supervised pretext tasks and later fine-tuned on the downstream target task.
  arXiv  Detail & Related papers  (2022-10-20T15:21:51Z)
• Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition [56.91538445510214]
  We present a learning algorithm for human activity recognition in videos. Our approach is designed for UAV videos, which are mainly acquired from obliquely placed dynamic cameras. We conduct extensive experiments on the UAV Human dataset and the NEC Drone dataset.
  arXiv  Detail & Related papers  (2022-09-15T22:16:52Z)
• CIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition [80.07590100872548]
  We propose Contrastive Inhibitory Adaptati On (CIAO), a mechanism that adapts the last layer of facial encoders to depict specific affective characteristics on different datasets. CIAO presents an improvement in facial expression recognition performance over six different datasets with very unique affective representations.
  arXiv  Detail & Related papers  (2022-08-10T15:46:05Z)
• Frame-level Prediction of Facial Expressions, Valence, Arousal and Action Units for Mobile Devices [7.056222499095849]
  We propose the novel frame-level emotion recognition algorithm by extracting facial features with the single EfficientNet model pre-trained on AffectNet. Our approach may be implemented even for video analytics on mobile devices.
  arXiv  Detail & Related papers  (2022-03-25T03:53:27Z)
• Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection [112.96004727646115]
  We develop a method to detect face-manipulated videos using real talking faces. We show that our method achieves state-of-the-art performance on cross-manipulation generalisation and robustness experiments. Our results suggest that leveraging natural and unlabelled videos is a promising direction for the development of more robust face forgery detectors.
  arXiv  Detail & Related papers  (2022-01-18T17:14:54Z)
• Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition [9.188777864190204]
  We introduce our submission to the 2nd Affective Behavior Analysis in-the-wild (ABAW2) Competition. In dealing with different emotion representations, we propose a multi-task streaming network. We leverage an advanced facial expression embedding as prior knowledge.
  arXiv  Detail & Related papers  (2021-07-08T09:35:08Z)
• Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework [83.21732533130846]
  The paper focuses on large in-the-wild databases, i.e., Aff-Wild and Aff-Wild2. It presents the design of two classes of deep neural networks trained with these databases. A novel multi-task and holistic framework is presented which is able to jointly learn and effectively generalize and perform affect recognition.
  arXiv  Detail & Related papers  (2021-03-29T17:36:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.