EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributes
- URL: http://arxiv.org/abs/2307.07961v2
- Date: Fri, 28 Jul 2023 15:38:19 GMT
- Title: EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributes
- Authors: Jingyuan Yang, Qirui Huang, Tingting Ding, Dani Lischinski, Daniel
Cohen-Or, Hui Huang
- Abstract summary: We introduce EmoSet, the first large-scale visual emotion dataset annotated with rich attributes.
EmoSet comprises 3.3 million images in total, with 118,102 of these images carefully labeled by human annotators.
Motivated by psychological studies, in addition to emotion category, each image is also annotated with a set of describable emotion attributes.
- Score: 53.95428298229396
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Visual Emotion Analysis (VEA) aims at predicting people's emotional responses
to visual stimuli. This is a promising, yet challenging, task in affective
computing, which has drawn increasing attention in recent years. Most of the
existing work in this area focuses on feature design, while little attention
has been paid to dataset construction. In this work, we introduce EmoSet, the
first large-scale visual emotion dataset annotated with rich attributes, which
is superior to existing datasets in four aspects: scale, annotation richness,
diversity, and data balance. EmoSet comprises 3.3 million images in total, with
118,102 of these images carefully labeled by human annotators, making it five
times larger than the largest existing dataset. EmoSet includes images from
social networks, as well as artistic images, and it is well balanced between
different emotion categories. Motivated by psychological studies, in addition
to emotion category, each image is also annotated with a set of describable
emotion attributes: brightness, colorfulness, scene type, object class, facial
expression, and human action, which can help understand visual emotions in a
precise and interpretable way. The relevance of these emotion attributes is
validated by analyzing the correlations between them and visual emotion, as
well as by designing an attribute module to help visual emotion recognition. We
believe EmoSet will bring some key insights and encourage further research in
visual emotion analysis and understanding. Project page:
https://vcc.tech/EmoSet.
Related papers
- UniEmoX: Cross-modal Semantic-Guided Large-Scale Pretraining for Universal Scene Emotion Perception [8.54013419046987]
We introduce UniEmoX, a cross-modal semantic-guided large-scale pretraining framework for visual emotion analysis.
By exploiting the similarity between paired and unpaired image-text samples, UniEmoX distills rich semantic knowledge from the CLIP model to enhance emotional embedding representations.
We develop a visual emotional dataset titled Emo8, covering nearly all common emotional scenes.
arXiv Detail & Related papers (2024-09-27T16:12:51Z) - EmoLLM: Multimodal Emotional Understanding Meets Large Language Models [61.179731667080326]
Multi-modal large language models (MLLMs) have achieved remarkable performance on objective multimodal perception tasks.
But their ability to interpret subjective, emotionally nuanced multimodal content remains largely unexplored.
EmoLLM is a novel model for multimodal emotional understanding, incorporating with two core techniques.
arXiv Detail & Related papers (2024-06-24T08:33:02Z) - Language Models (Mostly) Do Not Consider Emotion Triggers When Predicting Emotion [87.18073195745914]
We investigate how well human-annotated emotion triggers correlate with features deemed salient in their prediction of emotions.
Using EmoTrigger, we evaluate the ability of large language models to identify emotion triggers.
Our analysis reveals that emotion triggers are largely not considered salient features for emotion prediction models, instead there is intricate interplay between various features and the task of emotion detection.
arXiv Detail & Related papers (2023-11-16T06:20:13Z) - SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network [83.27291945217424]
We propose a novel Scene-Object interreLated Visual Emotion Reasoning network (SOLVER) to predict emotions from images.
To mine the emotional relationships between distinct objects, we first build up an Emotion Graph based on semantic concepts and visual features.
We also design a Scene-Object Fusion Module to integrate scenes and objects, which exploits scene features to guide the fusion process of object features with the proposed scene-based attention mechanism.
arXiv Detail & Related papers (2021-10-24T02:41:41Z) - Affective Image Content Analysis: Two Decades Review and New
Perspectives [132.889649256384]
We will comprehensively review the development of affective image content analysis (AICA) in the recent two decades.
We will focus on the state-of-the-art methods with respect to three main challenges -- the affective gap, perception subjectivity, and label noise and absence.
We discuss some challenges and promising research directions in the future, such as image content and context understanding, group emotion clustering, and viewer-image interaction.
arXiv Detail & Related papers (2021-06-30T15:20:56Z) - A Circular-Structured Representation for Visual Emotion Distribution
Learning [82.89776298753661]
We propose a well-grounded circular-structured representation to utilize the prior knowledge for visual emotion distribution learning.
To be specific, we first construct an Emotion Circle to unify any emotional state within it.
On the proposed Emotion Circle, each emotion distribution is represented with an emotion vector, which is defined with three attributes.
arXiv Detail & Related papers (2021-06-23T14:53:27Z) - Context Based Emotion Recognition using EMOTIC Dataset [22.631542327834595]
We present EMOTIC, a dataset of images of people annotated with their apparent emotion.
Using the EMOTIC dataset we train different CNN models for emotion recognition.
Our results show how scene context provides important information to automatically recognize emotional states.
arXiv Detail & Related papers (2020-03-30T12:38:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.