Domain Adaptation based Interpretable Image Emotion Recognition using
Facial Expression Recognition
- URL: http://arxiv.org/abs/2011.08388v2
- Date: Wed, 7 Feb 2024 13:23:08 GMT
- Title: Domain Adaptation based Interpretable Image Emotion Recognition using
Facial Expression Recognition
- Authors: Puneet Kumar and Balasubramanian Raman
- Abstract summary: A domain adaptation technique has been proposed in this paper to identify the emotions in generic images containing facial & non-facial objects and non-human components.
It addresses the challenge of the insufficient availability of pre-trained models and well-annotated datasets for image emotion recognition (IER)
It starts with proposing a facial emotion recognition (FER) system and then moves on to adapting it for image emotion recognition.
- Score: 13.950116138578267
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: A domain adaptation technique has been proposed in this paper to identify the
emotions in generic images containing facial & non-facial objects and non-human
components. It addresses the challenge of the insufficient availability of
pre-trained models and well-annotated datasets for image emotion recognition
(IER). It starts with proposing a facial emotion recognition (FER) system and
then moves on to adapting it for image emotion recognition. First, a
deep-learning-based FER system has been proposed that classifies a given facial
image into discrete emotion classes. Further, an image recognition system has
been proposed that adapts the proposed FER system to recognize the emotions
portrayed by images using domain adaptation. It classifies the generic images
into 'happy,' 'sad,' 'hate,' and 'anger' classes. A novel interpretability
approach, Divide and Conquer based Shap (DnCShap), has also been proposed to
interpret the highly relevant visual features for emotion recognition. The
proposed system's architecture has been decided through ablation studies, and
the experiments are conducted on four FER and four IER datasets. The proposed
IER system has shown an emotion classification accuracy of 59.61% for the IAPSa
dataset, 57.83% for the ArtPhoto dataset, 67.93% for the FI dataset, and 55.13%
for the EMOTIC dataset. The important visual features leading to a particular
emotion class have been identified, and the embedding plots for various emotion
classes have been analyzed to explain the proposed system's predictions.
Related papers
- Alleviating Catastrophic Forgetting in Facial Expression Recognition with Emotion-Centered Models [49.3179290313959]
The proposed method, emotion-centered generative replay (ECgr), tackles this challenge by integrating synthetic images from generative adversarial networks.
ECgr incorporates a quality assurance algorithm to ensure the fidelity of generated images.
The experimental results on four diverse facial expression datasets demonstrate that incorporating images generated by our pseudo-rehearsal method enhances training on the targeted dataset and the source dataset.
arXiv Detail & Related papers (2024-04-18T15:28:34Z) - Music Recommendation Based on Facial Emotion Recognition [0.0]
This paper presents a comprehensive approach to enhancing the user experience through the integration of emotion recognition, music recommendation, and explainable AI using GRAD-CAM.
The proposed methodology utilizes a ResNet50 model trained on the Facial Expression Recognition dataset, consisting of real images of individuals expressing various emotions.
arXiv Detail & Related papers (2024-04-06T15:14:25Z) - StyleEDL: Style-Guided High-order Attention Network for Image Emotion
Distribution Learning [69.06749934902464]
We propose a style-guided high-order attention network for image emotion distribution learning termed StyleEDL.
StyleEDL interactively learns stylistic-aware representations of images by exploring the hierarchical stylistic information of visual contents.
In addition, we introduce a stylistic graph convolutional network to dynamically generate the content-dependent emotion representations.
arXiv Detail & Related papers (2023-08-06T03:22:46Z) - SAFER: Situation Aware Facial Emotion Recognition [0.0]
We present SAFER, a novel system for emotion recognition from facial expressions.
It employs state-of-the-art deep learning techniques to extract various features from facial images.
It can adapt to unseen and varied facial expressions, making it suitable for real-world applications.
arXiv Detail & Related papers (2023-06-14T20:42:26Z) - PERI: Part Aware Emotion Recognition In The Wild [4.206175795966693]
This paper focuses on emotion recognition using visual features.
We create part aware spatial (PAS) images by extracting key regions from the input image using a mask generated from both body pose and facial landmarks.
We provide our results on the publicly available in the wild EMOTIC dataset.
arXiv Detail & Related papers (2022-10-18T20:01:40Z) - Interpretable Multimodal Emotion Recognition using Hybrid Fusion of
Speech and Image Data [15.676632465869346]
A new interpretability technique has been developed to identify the important speech & image features leading to the prediction of particular emotion classes.
The proposed system has achieved 83.29% accuracy for emotion recognition.
arXiv Detail & Related papers (2022-08-25T04:43:34Z) - Emotion Separation and Recognition from a Facial Expression by
Generating the Poker Face with Vision Transformers [57.67586172996843]
We propose a novel FER model, called Poker Face Vision Transformer or PF-ViT, to separate and recognize the disturbance-agnostic emotion from a static facial image.
PF-ViT generates its corresponding poker face without the need for paired images.
arXiv Detail & Related papers (2022-07-22T13:39:06Z) - SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network [83.27291945217424]
We propose a novel Scene-Object interreLated Visual Emotion Reasoning network (SOLVER) to predict emotions from images.
To mine the emotional relationships between distinct objects, we first build up an Emotion Graph based on semantic concepts and visual features.
We also design a Scene-Object Fusion Module to integrate scenes and objects, which exploits scene features to guide the fusion process of object features with the proposed scene-based attention mechanism.
arXiv Detail & Related papers (2021-10-24T02:41:41Z) - Affective Image Content Analysis: Two Decades Review and New
Perspectives [132.889649256384]
We will comprehensively review the development of affective image content analysis (AICA) in the recent two decades.
We will focus on the state-of-the-art methods with respect to three main challenges -- the affective gap, perception subjectivity, and label noise and absence.
We discuss some challenges and promising research directions in the future, such as image content and context understanding, group emotion clustering, and viewer-image interaction.
arXiv Detail & Related papers (2021-06-30T15:20:56Z) - A Circular-Structured Representation for Visual Emotion Distribution
Learning [82.89776298753661]
We propose a well-grounded circular-structured representation to utilize the prior knowledge for visual emotion distribution learning.
To be specific, we first construct an Emotion Circle to unify any emotional state within it.
On the proposed Emotion Circle, each emotion distribution is represented with an emotion vector, which is defined with three attributes.
arXiv Detail & Related papers (2021-06-23T14:53:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.