A Generalized Zero-Shot Framework for Emotion Recognition from Body Gestures

• URL: http://arxiv.org/abs/2010.06362v2
• Date: Tue, 20 Oct 2020 08:15:45 GMT
• Title: A Generalized Zero-Shot Framework for Emotion Recognition from Body Gestures
• Authors: Jinting Wu, Yujia Zhang, Xiaoguang Zhao and Wenbin Gao
• Abstract summary: We introduce a Generalized Zero-Shot Learning (GZSL) framework to infer the emotional state of new body gestures. The framework is significantly superior to the traditional method of emotion classification and state-of-the-art zero-shot learning methods.
• Score: 5.331671302839567
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although automatic emotion recognition from facial expressions and speech has made remarkable progress, emotion recognition from body gestures has not been thoroughly explored. People often use a variety of body language to express emotions, and it is difficult to enumerate all emotional body gestures and collect enough samples for each category. Therefore, recognizing new emotional body gestures is critical for better understanding human emotions. However, the existing methods fail to accurately determine which emotional state a new body gesture belongs to. In order to solve this problem, we introduce a Generalized Zero-Shot Learning (GZSL) framework, which consists of three branches to infer the emotional state of the new body gestures with only their semantic descriptions. The first branch is a Prototype-Based Detector (PBD) which is used to determine whether an sample belongs to a seen body gesture category and obtain the prediction results of the samples from the seen categories. The second branch is a Stacked AutoEncoder (StAE) with manifold regularization, which utilizes semantic representations to predict samples from unseen categories. Note that both of the above branches are for body gesture recognition. We further add an emotion classifier with a softmax layer as the third branch in order to better learn the feature representations for this emotion classification task. The input features for these three branches are learned by a shared feature extraction network, i.e., a Bidirectional Long Short-Term Memory Networks (BLSTM) with a self-attention module. We treat these three branches as subtasks and use multi-task learning strategies for joint training. The performance of our framework on an emotion recognition dataset is significantly superior to the traditional method of emotion classification and state-of-the-art zero-shot learning methods.

Related papers

• TONE: A 3-Tiered ONtology for Emotion analysis [9.227164881235947]
  Emotions have played an important part in many sectors, including psychology, medicine, mental health, computer science, and so on. Emotions can be classified using the following two methods: (1) The supervised method's efficiency is strongly dependent on the size and domain of the data collected. We create an emotion-based ontology that effectively creates an emotional hierarchy based on Dr. Gerrod Parrot's group of emotions.
  arXiv  Detail & Related papers  (2024-01-11T04:23:08Z)
• Seeking Subjectivity in Visual Emotion Distribution Learning [93.96205258496697]
  Visual Emotion Analysis (VEA) aims to predict people's emotions towards different visual stimuli. Existing methods often predict visual emotion distribution in a unified network, neglecting the inherent subjectivity in its crowd voting process. We propose a novel textitSubjectivity Appraise-and-Match Network (SAMNet) to investigate the subjectivity in visual emotion distribution.
  arXiv  Detail & Related papers  (2022-07-25T02:20:03Z)
• When Facial Expression Recognition Meets Few-Shot Learning: A Joint and Alternate Learning Framework [60.51225419301642]
  We propose an Emotion Guided Similarity Network (EGS-Net) to address the diversity of human emotions in practical scenarios. EGS-Net consists of an emotion branch and a similarity branch, based on a two-stage learning framework. Experimental results on both in-the-lab and in-the-wild compound expression datasets demonstrate the superiority of our proposed method against several state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-01-18T07:24:12Z)
• Stimuli-Aware Visual Emotion Analysis [75.68305830514007]
  We propose a stimuli-aware visual emotion analysis (VEA) method consisting of three stages, namely stimuli selection, feature extraction and emotion prediction. To the best of our knowledge, it is the first time to introduce stimuli selection process into VEA in an end-to-end network. Experiments demonstrate that the proposed method consistently outperforms the state-of-the-art approaches on four public visual emotion datasets.
  arXiv  Detail & Related papers  (2021-09-04T08:14:52Z)
• A Circular-Structured Representation for Visual Emotion Distribution Learning [82.89776298753661]
  We propose a well-grounded circular-structured representation to utilize the prior knowledge for visual emotion distribution learning. To be specific, we first construct an Emotion Circle to unify any emotional state within it. On the proposed Emotion Circle, each emotion distribution is represented with an emotion vector, which is defined with three attributes.
  arXiv  Detail & Related papers  (2021-06-23T14:53:27Z)
• Multi-Classifier Interactive Learning for Ambiguous Speech Emotion Recognition [9.856709988128515]
  We propose a novel multi-classifier interactive learning (MCIL) method to address the ambiguous speech emotions. MCIL mimics several individuals, who have inconsistent cognitions of ambiguous emotions, and construct new ambiguous labels. Experiments show that MCIL does not only improve each classifier's performance, but also raises their recognition consistency from moderate to substantial.
  arXiv  Detail & Related papers  (2020-12-10T02:58:34Z)
• Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition [55.44502358463217]
  We propose a modality-transferable model with emotion embeddings to tackle the aforementioned issues. Our model achieves state-of-the-art performance on most of the emotion categories. Our model also outperforms existing baselines in the zero-shot and few-shot scenarios for unseen emotions.
  arXiv  Detail & Related papers  (2020-09-21T06:10:39Z)
• Learning Unseen Emotions from Gestures via Semantically-Conditioned Zero-Shot Perception with Adversarial Autoencoders [25.774235606472875]
  We introduce an adversarial, autoencoder-based representation learning that correlates 3D motion-captured gesture sequence with the vectorized representation of the natural-language perceived emotion terms. We train our method using a combination of gestures annotated with known emotion terms and gestures not annotated with any emotions.
  arXiv  Detail & Related papers  (2020-09-18T15:59:44Z)
• Emotion Recognition From Gait Analyses: Current Research and Future Directions [48.93172413752614]
  gait conveys information about the walker's emotion. The mapping between various emotions and gait patterns provides a new source for automated emotion recognition. gait is remotely observable, more difficult to imitate, and requires less cooperation from the subject.
  arXiv  Detail & Related papers  (2020-03-13T08:22:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.