Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion

• URL: http://arxiv.org/abs/2307.13463v1
• Date: Tue, 25 Jul 2023 12:47:21 GMT
• Title: Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion
• Authors: James Z. Wang, Sicheng Zhao, Chenyan Wu, Reginald B. Adams, Michelle G. Newman, Tal Shafir, Rachelle Tsachor
• Abstract summary: This article provides a comprehensive overview of the field of emotion analysis in visual media. We discuss the psychological foundations of emotion and the computational principles that underpin the understanding of emotions from images and videos. We contend that this represents a "Holy Grail" research problem in computing and delineate pivotal directions for future inquiry.
• Score: 24.920797480215242
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The emergence of artificial emotional intelligence technology is revolutionizing the fields of computers and robotics, allowing for a new level of communication and understanding of human behavior that was once thought impossible. While recent advancements in deep learning have transformed the field of computer vision, automated understanding of evoked or expressed emotions in visual media remains in its infancy. This foundering stems from the absence of a universally accepted definition of "emotion", coupled with the inherently subjective nature of emotions and their intricate nuances. In this article, we provide a comprehensive, multidisciplinary overview of the field of emotion analysis in visual media, drawing on insights from psychology, engineering, and the arts. We begin by exploring the psychological foundations of emotion and the computational principles that underpin the understanding of emotions from images and videos. We then review the latest research and systems within the field, accentuating the most promising approaches. We also discuss the current technological challenges and limitations of emotion analysis, underscoring the necessity for continued investigation and innovation. We contend that this represents a "Holy Grail" research problem in computing and delineate pivotal directions for future inquiry. Finally, we examine the ethical ramifications of emotion-understanding technologies and contemplate their potential societal impacts. Overall, this article endeavors to equip readers with a deeper understanding of the domain of emotion analysis in visual media and to inspire further research and development in this captivating and rapidly evolving field.

Related papers

• Emotion Detection through Body Gesture and Face [0.0]
  The project addresses the challenge of emotion recognition by focusing on non-facial cues, specifically hands, body gestures, and gestures. Traditional emotion recognition systems mainly rely on facial expression analysis and often ignore the rich emotional information conveyed through body language. The project aims to contribute to the field of affective computing by enhancing the ability of machines to interpret and respond to human emotions in a more comprehensive and nuanced way.
  arXiv  Detail & Related papers  (2024-07-13T15:15:50Z)
• Generative Technology for Human Emotion Recognition: A Scope Review [11.578408396744237]
  This survey aims to bridge the gaps in the existing literature by conducting a comprehensive analysis of over 320 research papers until June 2024. It will introduce the mathematical principles of different generative models and the commonly used datasets. It will provide an in-depth analysis of how generative techniques address emotion recognition based on different modalities.
  arXiv  Detail & Related papers  (2024-07-04T05:22:55Z)
• Bodily expressed emotion understanding through integrating Laban movement analysis [7.73546354173679]
  This study develops a high-quality human motor element dataset based on the Laban Movement Analysis movement coding system. Our long-term ambition is to integrate knowledge from computing, psychology, and performing arts to enable automated understanding and analysis of emotion and mental state through body language.
  arXiv  Detail & Related papers  (2023-04-05T02:07:15Z)
• SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network [83.27291945217424]
  We propose a novel Scene-Object interreLated Visual Emotion Reasoning network (SOLVER) to predict emotions from images. To mine the emotional relationships between distinct objects, we first build up an Emotion Graph based on semantic concepts and visual features. We also design a Scene-Object Fusion Module to integrate scenes and objects, which exploits scene features to guide the fusion process of object features with the proposed scene-based attention mechanism.
  arXiv  Detail & Related papers  (2021-10-24T02:41:41Z)
• Emotion Recognition from Multiple Modalities: Fundamentals and Methodologies [106.62835060095532]
  We discuss several key aspects of multi-modal emotion recognition (MER) We begin with a brief introduction on widely used emotion representation models and affective modalities. We then summarize existing emotion annotation strategies and corresponding computational tasks. Finally, we outline several real-world applications and discuss some future directions.
  arXiv  Detail & Related papers  (2021-08-18T21:55:20Z)
• Emotion Recognition for Healthcare Surveillance Systems Using Neural Networks: A Survey [8.31246680772592]
  We present recent research in the field of using neural networks to recognize emotions. We focus on studying emotions' recognition from speech, facial expressions, and audio-visual input. These three emotion recognition techniques can be used as a surveillance system in healthcare centers to monitor patients.
  arXiv  Detail & Related papers  (2021-07-13T11:17:00Z)
• Affective Image Content Analysis: Two Decades Review and New Perspectives [132.889649256384]
  We will comprehensively review the development of affective image content analysis (AICA) in the recent two decades. We will focus on the state-of-the-art methods with respect to three main challenges -- the affective gap, perception subjectivity, and label noise and absence. We discuss some challenges and promising research directions in the future, such as image content and context understanding, group emotion clustering, and viewer-image interaction.
  arXiv  Detail & Related papers  (2021-06-30T15:20:56Z)
• Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction [55.47134146639492]
  This article proposes a unifed end-to-end neural architecture, which is capable of simultaneously encoding the semantics and the emotions in a post. Experiments on real-world data demonstrate that the proposed method outperforms the state-of-the-art methods in terms of both content coherence and emotion appropriateness.
  arXiv  Detail & Related papers  (2021-06-06T06:26:15Z)
• Computational Emotion Analysis From Images: Recent Advances and Future Directions [79.05003998727103]
  In this chapter, we aim to introduce image emotion analysis (IEA) from a computational perspective. We begin with commonly used emotion representation models from psychology. We then define the key computational problems that the researchers have been trying to solve.
  arXiv  Detail & Related papers  (2021-03-19T13:33:34Z)
• Knowledge Bridging for Empathetic Dialogue Generation [52.39868458154947]
  Lack of external knowledge makes empathetic dialogue systems difficult to perceive implicit emotions and learn emotional interactions from limited dialogue history. We propose to leverage external knowledge, including commonsense knowledge and emotional lexical knowledge, to explicitly understand and express emotions in empathetic dialogue generation.
  arXiv  Detail & Related papers  (2020-09-21T09:21:52Z)
• Sentiment Analysis: Automatically Detecting Valence, Emotions, and Other Affectual States from Text [31.87319293259599]
  This article presents a sweeping overview of sentiment analysis research. It includes the origins of the field, the rich landscape of tasks, challenges, a survey of the methods and resources used, and applications. We discuss how, without careful fore-thought, sentiment analysis has the potential for harmful outcomes.
  arXiv  Detail & Related papers  (2020-05-25T01:37:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.