Affective Behaviour Analysis via Progressive Learning

• URL: http://arxiv.org/abs/2407.16945v2
• Date: Fri, 26 Jul 2024 02:24:11 GMT
• Title: Affective Behaviour Analysis via Progressive Learning
• Authors: Chen Liu, Wei Zhang, Feng Qiu, Lincheng Li, Xin Yu,
• Abstract summary: We present our methods and experimental results for the two competition tracks. We train a Masked-Auto in a self-supervised manner to attain high-quality facial features. We utilize curriculum learning to transition the model from recognizing single expressions to recognizing compound expressions.
• Score: 23.455163723584427
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Affective Behavior Analysis aims to develop emotionally intelligent technology that can recognize and respond to human emotions. To advance this, the 7th Affective Behavior Analysis in-the-wild (ABAW) competition establishes two tracks: i.e., the Multi-task Learning (MTL) Challenge and the Compound Expression (CE) challenge based on Aff-Wild2 and C-EXPR-DB datasets. In this paper, we present our methods and experimental results for the two competition tracks. Specifically, it can be summarized in the following four aspects: 1) To attain high-quality facial features, we train a Masked-Auto Encoder in a self-supervised manner. 2) We devise a temporal convergence module to capture the temporal information between video frames and explore the impact of window size and sequence length on each sub-task. 3) To facilitate the joint optimization of various sub-tasks, we explore the impact of sub-task joint training and feature fusion from individual tasks on each task performance improvement. 4) We utilize curriculum learning to transition the model from recognizing single expressions to recognizing compound expressions, thereby improving the accuracy of compound expression recognition. Extensive experiments demonstrate the superiority of our designs.

Related papers

• Two in One Go: Single-stage Emotion Recognition with Decoupled Subject-context Transformer [78.35816158511523]
  We present a single-stage emotion recognition approach, employing a Decoupled Subject-Context Transformer (DSCT) for simultaneous subject localization and emotion classification. We evaluate our single-stage framework on two widely used context-aware emotion recognition datasets, CAER-S and EMOTIC.
  arXiv  Detail & Related papers  (2024-04-26T07:30:32Z)
• Affective Behaviour Analysis via Integrating Multi-Modal Knowledge [24.74463315135503]
  The 6th competition on Affective Behavior Analysis in-the-wild (ABAW) utilizes the Aff-Wild2, Hume-Vidmimic2, and C-EXPR-DB datasets. We present our method designs for the five competitive tracks, i.e., Valence-Arousal (VA) Estimation, Expression (EXPR) Recognition, Action Unit (AU) Detection, Compound Expression (CE) Recognition, and Emotional Mimicry Intensity (EMI) Estimation.
  arXiv  Detail & Related papers  (2024-03-16T06:26:43Z)
• The 6th Affective Behavior Analysis in-the-wild (ABAW) Competition [53.718777420180395]
  This paper describes the 6th Affective Behavior Analysis in-the-wild (ABAW) Competition. The 6th ABAW Competition addresses contemporary challenges in understanding human emotions and behaviors.
  arXiv  Detail & Related papers  (2024-02-29T16:49:38Z)
• Multimodal Visual-Tactile Representation Learning through Self-Supervised Contrastive Pre-Training [0.850206009406913]
  MViTac is a novel methodology that leverages contrastive learning to integrate vision and touch sensations in a self-supervised fashion. By availing both sensory inputs, MViTac leverages intra and inter-modality losses for learning representations, resulting in enhanced material property classification and more adept grasping prediction.
  arXiv  Detail & Related papers  (2024-01-22T15:11:57Z)
• Disentangled Interaction Representation for One-Stage Human-Object Interaction Detection [70.96299509159981]
  Human-Object Interaction (HOI) detection is a core task for human-centric image understanding. Recent one-stage methods adopt a transformer decoder to collect image-wide cues that are useful for interaction prediction. Traditional two-stage methods benefit significantly from their ability to compose interaction features in a disentangled and explainable manner.
  arXiv  Detail & Related papers  (2023-12-04T08:02:59Z)
• Re-mine, Learn and Reason: Exploring the Cross-modal Semantic Correlations for Language-guided HOI detection [57.13665112065285]
  Human-Object Interaction (HOI) detection is a challenging computer vision task. We present a framework that enhances HOI detection by incorporating structured text knowledge.
  arXiv  Detail & Related papers  (2023-07-25T14:20:52Z)
• Task Formulation Matters When Learning Continually: A Case Study in Visual Question Answering [58.82325933356066]
  Continual learning aims to train a model incrementally on a sequence of tasks without forgetting previous knowledge. We present a detailed study of how different settings affect performance for Visual Question Answering.
  arXiv  Detail & Related papers  (2022-09-30T19:12:58Z)
• Multi-task Cross Attention Network in Facial Behavior Analysis [7.910908058662372]
  We present our solution for the Multi-Task Learning challenge of the Affective Behavior Analysis in-the-wild competition. The challenge is a combination of three tasks: action unit detection, facial expression recognition and valance-arousal estimation. We introduce a cross-attentive module to improve multi-task learning performance.
  arXiv  Detail & Related papers  (2022-07-21T04:07:07Z)
• Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition [9.188777864190204]
  We introduce our submission to the 2nd Affective Behavior Analysis in-the-wild (ABAW2) Competition. In dealing with different emotion representations, we propose a multi-task streaming network. We leverage an advanced facial expression embedding as prior knowledge.
  arXiv  Detail & Related papers  (2021-07-08T09:35:08Z)
• A Multi-resolution Approach to Expression Recognition in the Wild [9.118706387430883]
  We propose a multi-resolution approach to solve the Facial Expression Recognition task. We ground our intuition on the observation that often faces images are acquired at different resolutions. To our aim, we use a ResNet-like architecture, equipped with Squeeze-and-Excitation blocks, trained on the Affect-in-the-Wild 2 dataset.
  arXiv  Detail & Related papers  (2021-03-09T21:21:02Z)
• Learning Modality Interaction for Temporal Sentence Localization and Event Captioning in Videos [76.21297023629589]
  We propose a novel method for learning pairwise modality interactions in order to better exploit complementary information for each pair of modalities in videos. Our method turns out to achieve state-of-the-art performances on four standard benchmark datasets.
  arXiv  Detail & Related papers  (2020-07-28T12:40:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.