Facial Emotion Recognition with Noisy Multi-task Annotations

• URL: http://arxiv.org/abs/2010.09849v2
• Date: Tue, 24 Nov 2020 17:43:01 GMT
• Title: Facial Emotion Recognition with Noisy Multi-task Annotations
• Authors: Siwei Zhang, Zhiwu Huang, Danda Pani Paudel, Luc Van Gool
• Abstract summary: We introduce a new problem of facial emotion recognition with noisy multi-task annotations. For this new problem, we suggest a formulation from the point of joint distribution match view. We exploit a new method to enable the emotion prediction and the joint distribution learning.
• Score: 88.42023952684052
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human emotions can be inferred from facial expressions. However, the annotations of facial expressions are often highly noisy in common emotion coding models, including categorical and dimensional ones. To reduce human labelling effort on multi-task labels, we introduce a new problem of facial emotion recognition with noisy multi-task annotations. For this new problem, we suggest a formulation from the point of joint distribution match view, which aims at learning more reliable correlations among raw facial images and multi-task labels, resulting in the reduction of noise influence. In our formulation, we exploit a new method to enable the emotion prediction and the joint distribution learning in a unified adversarial learning game. Evaluation throughout extensive experiments studies the real setups of the suggested new problem, as well as the clear superiority of the proposed method over the state-of-the-art competing methods on either the synthetic noisy labeled CIFAR-10 or practical noisy multi-task labeled RAF and AffectNet. The code is available at https://github.com/sanweiliti/noisyFER.

Related papers

• Knowledge-Enhanced Facial Expression Recognition with Emotional-to-Neutral Transformation [66.53435569574135]
  Existing facial expression recognition methods typically fine-tune a pre-trained visual encoder using discrete labels. We observe that the rich knowledge in text embeddings, generated by vision-language models, is a promising alternative for learning discriminative facial expression representations. We propose a novel knowledge-enhanced FER method with an emotional-to-neutral transformation.
  arXiv  Detail & Related papers  (2024-09-13T07:28:57Z)
• Bridging Modalities: Knowledge Distillation and Masked Training for Translating Multi-Modal Emotion Recognition to Uni-Modal, Speech-Only Emotion Recognition [0.0]
  This paper presents an innovative approach to address the challenges of translating multi-modal emotion recognition models to a more practical uni-modal counterpart. Recognizing emotions from speech signals is a critical task with applications in human-computer interaction, affective computing, and mental health assessment.
  arXiv  Detail & Related papers  (2024-01-04T22:42:14Z)
• Seeking Subjectivity in Visual Emotion Distribution Learning [93.96205258496697]
  Visual Emotion Analysis (VEA) aims to predict people's emotions towards different visual stimuli. Existing methods often predict visual emotion distribution in a unified network, neglecting the inherent subjectivity in its crowd voting process. We propose a novel textitSubjectivity Appraise-and-Match Network (SAMNet) to investigate the subjectivity in visual emotion distribution.
  arXiv  Detail & Related papers  (2022-07-25T02:20:03Z)
• Learn From All: Erasing Attention Consistency for Noisy Label Facial Expression Recognition [37.83484824542302]
  We explore dealing with noisy labels from a new feature-learning perspective. We find that FER models remember noisy samples by focusing on a part of the features that can be considered related to the noisy labels. Inspired by that, we propose a novel Erasing Attention Consistency (EAC) method to suppress the noisy samples.
  arXiv  Detail & Related papers  (2022-07-21T04:30:33Z)
• Affect-DML: Context-Aware One-Shot Recognition of Human Affect using Deep Metric Learning [29.262204241732565]
  Existing methods assume that all emotions-of-interest are given a priori as annotated training examples. We conceptualize one-shot recognition of emotions in context -- a new problem aimed at recognizing human affect states in finer particle level from a single support sample. All variants of our model clearly outperform the random baseline, while leveraging the semantic scene context consistently improves the learnt representations.
  arXiv  Detail & Related papers  (2021-11-30T10:35:20Z)
• Label Distribution Amendment with Emotional Semantic Correlations for Facial Expression Recognition [69.18918567657757]
  We propose a new method that amends the label distribution of each facial image by leveraging correlations among expressions in the semantic space. By comparing semantic and task class-relation graphs of each image, the confidence of its label distribution is evaluated. Experimental results demonstrate the proposed method is more effective than compared state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-07-23T07:46:14Z)
• Multi-Classifier Interactive Learning for Ambiguous Speech Emotion Recognition [9.856709988128515]
  We propose a novel multi-classifier interactive learning (MCIL) method to address the ambiguous speech emotions. MCIL mimics several individuals, who have inconsistent cognitions of ambiguous emotions, and construct new ambiguous labels. Experiments show that MCIL does not only improve each classifier's performance, but also raises their recognition consistency from moderate to substantial.
  arXiv  Detail & Related papers  (2020-12-10T02:58:34Z)
• Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition [55.44502358463217]
  We propose a modality-transferable model with emotion embeddings to tackle the aforementioned issues. Our model achieves state-of-the-art performance on most of the emotion categories. Our model also outperforms existing baselines in the zero-shot and few-shot scenarios for unseen emotions.
  arXiv  Detail & Related papers  (2020-09-21T06:10:39Z)
• Deep Multi-task Learning for Facial Expression Recognition and Synthesis Based on Selective Feature Sharing [28.178390846446938]
  We propose a novel selective feature-sharing method, and establish a multi-task network for facial expression recognition and facial expression synthesis. The proposed method can effectively transfer beneficial features between different tasks, while filtering out useless and harmful information. Experimental results show that the proposed method achieves state-of-the-art performance on those commonly used facial expression recognition benchmarks.
  arXiv  Detail & Related papers  (2020-07-09T02:29:34Z)
• Learning to Augment Expressions for Few-shot Fine-grained Facial Expression Recognition [98.83578105374535]
  We present a novel Fine-grained Facial Expression Database - F2ED. It includes more than 200k images with 54 facial expressions from 119 persons. Considering the phenomenon of uneven data distribution and lack of samples is common in real-world scenarios, we evaluate several tasks of few-shot expression learning. We propose a unified task-driven framework - Compositional Generative Adversarial Network (Comp-GAN) learning to synthesize facial images.
  arXiv  Detail & Related papers  (2020-01-17T03:26:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.