Multi-Branch Network for Imagery Emotion Prediction
- URL: http://arxiv.org/abs/2312.07500v1
- Date: Tue, 12 Dec 2023 18:34:56 GMT
- Title: Multi-Branch Network for Imagery Emotion Prediction
- Authors: Quoc-Bao Ninh, Hai-Chan Nguyen, Triet Huynh, Trung-Nghia Le
- Abstract summary: We present a novel Multi-Branch Network (MBN) to predict both discrete and continuous emotions in an image.
Our proposed method significantly outperforms state-of-the-art methods with 28.4% in mAP and 0.93 in MAE.
- Score: 4.618814297494939
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: For a long time, images have proved perfect at both storing and conveying
rich semantics, especially human emotions. A lot of research has been conducted
to provide machines with the ability to recognize emotions in photos of people.
Previous methods mostly focus on facial expressions but fail to consider the
scene context, meanwhile scene context plays an important role in predicting
emotions, leading to more accurate results. In addition,
Valence-Arousal-Dominance (VAD) values offer a more precise quantitative
understanding of continuous emotions, yet there has been less emphasis on
predicting them compared to discrete emotional categories. In this paper, we
present a novel Multi-Branch Network (MBN), which utilizes various source
information, including faces, bodies, and scene contexts to predict both
discrete and continuous emotions in an image. Experimental results on EMOTIC
dataset, which contains large-scale images of people in unconstrained
situations labeled with 26 discrete categories of emotions and VAD values, show
that our proposed method significantly outperforms state-of-the-art methods
with 28.4% in mAP and 0.93 in MAE. The results highlight the importance of
utilizing multiple contextual information in emotion prediction and illustrate
the potential of our proposed method in a wide range of applications, such as
effective computing, human-computer interaction, and social robotics. Source
code:
https://github.com/BaoNinh2808/Multi-Branch-Network-for-Imagery-Emotion-Prediction
Related papers
- Using Scene and Semantic Features for Multi-modal Emotion Recognition [1.0152838128195467]
We propose to use combined scene and semantic features, along with personal features, for multi-modal emotion recognition.
We use a modified EmbraceNet to extract features from the images, which is trained to learn both the body and pose features simultaneously.
We report an average precision of 40.39% across the 26 emotion categories, which is a 5% improvement over previous approaches.
arXiv Detail & Related papers (2023-08-01T01:54:55Z) - EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributes [53.95428298229396]
We introduce EmoSet, the first large-scale visual emotion dataset annotated with rich attributes.
EmoSet comprises 3.3 million images in total, with 118,102 of these images carefully labeled by human annotators.
Motivated by psychological studies, in addition to emotion category, each image is also annotated with a set of describable emotion attributes.
arXiv Detail & Related papers (2023-07-16T06:42:46Z) - High-Level Context Representation for Emotion Recognition in Images [4.987022981158291]
We propose an approach for high-level context representation extraction from images.
The model relies on a single cue and a single encoding stream to correlate this representation with emotions.
Our approach is more efficient than previous models and can be easily deployed to address real-world problems related to emotion recognition.
arXiv Detail & Related papers (2023-05-05T13:20:41Z) - Performance Analysis and Evaluation of Cloud Vision Emotion APIs [0.0]
Performance of two well-known APIs were compared using a public dataset of 980 images of facial emotions.
It has been found that the prediction accuracy for each emotion varies according to the cloud service being used.
arXiv Detail & Related papers (2023-03-23T00:47:43Z) - Multi-Cue Adaptive Emotion Recognition Network [4.570705738465714]
We propose a new deep learning approach for emotion recognition based on adaptive multi-cues.
We compare the proposed approach with the state-of-art approaches in the CAER-S dataset.
arXiv Detail & Related papers (2021-11-03T15:08:55Z) - SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network [83.27291945217424]
We propose a novel Scene-Object interreLated Visual Emotion Reasoning network (SOLVER) to predict emotions from images.
To mine the emotional relationships between distinct objects, we first build up an Emotion Graph based on semantic concepts and visual features.
We also design a Scene-Object Fusion Module to integrate scenes and objects, which exploits scene features to guide the fusion process of object features with the proposed scene-based attention mechanism.
arXiv Detail & Related papers (2021-10-24T02:41:41Z) - Stimuli-Aware Visual Emotion Analysis [75.68305830514007]
We propose a stimuli-aware visual emotion analysis (VEA) method consisting of three stages, namely stimuli selection, feature extraction and emotion prediction.
To the best of our knowledge, it is the first time to introduce stimuli selection process into VEA in an end-to-end network.
Experiments demonstrate that the proposed method consistently outperforms the state-of-the-art approaches on four public visual emotion datasets.
arXiv Detail & Related papers (2021-09-04T08:14:52Z) - Affective Image Content Analysis: Two Decades Review and New
Perspectives [132.889649256384]
We will comprehensively review the development of affective image content analysis (AICA) in the recent two decades.
We will focus on the state-of-the-art methods with respect to three main challenges -- the affective gap, perception subjectivity, and label noise and absence.
We discuss some challenges and promising research directions in the future, such as image content and context understanding, group emotion clustering, and viewer-image interaction.
arXiv Detail & Related papers (2021-06-30T15:20:56Z) - A Circular-Structured Representation for Visual Emotion Distribution
Learning [82.89776298753661]
We propose a well-grounded circular-structured representation to utilize the prior knowledge for visual emotion distribution learning.
To be specific, we first construct an Emotion Circle to unify any emotional state within it.
On the proposed Emotion Circle, each emotion distribution is represented with an emotion vector, which is defined with three attributes.
arXiv Detail & Related papers (2021-06-23T14:53:27Z) - Emotion-Based End-to-End Matching Between Image and Music in
Valence-Arousal Space [80.49156615923106]
Matching images and music with similar emotions might help to make emotion perceptions more vivid and stronger.
Existing emotion-based image and music matching methods either employ limited categorical emotion states or train the matching model using an impractical multi-stage pipeline.
In this paper, we study end-to-end matching between image and music based on emotions in the continuous valence-arousal (VA) space.
arXiv Detail & Related papers (2020-08-22T20:12:23Z) - Context Based Emotion Recognition using EMOTIC Dataset [22.631542327834595]
We present EMOTIC, a dataset of images of people annotated with their apparent emotion.
Using the EMOTIC dataset we train different CNN models for emotion recognition.
Our results show how scene context provides important information to automatically recognize emotional states.
arXiv Detail & Related papers (2020-03-30T12:38:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.