Multimodal Sentiment Analysis: Perceived vs Induced Sentiments

• URL: http://arxiv.org/abs/2312.07627v1
• Date: Tue, 12 Dec 2023 07:24:02 GMT
• Title: Multimodal Sentiment Analysis: Perceived vs Induced Sentiments
• Abstract summary: We propose a framework that integrates visual and textual features to predict the GIF sentiment. It also incorporates attributes including face emotion detection and OCR generated captions to capture the semantic aspects of the GIF. The developed classifier achieves an accuracy of 82.7% on Twitter GIFs.
• Score: 1.6574413179773761
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Social media has created a global network where people can easily access and exchange vast information. This information gives rise to a variety of opinions, reflecting both positive and negative viewpoints. GIFs stand out as a multimedia format offering a visually engaging way for users to communicate. In this research, we propose a multimodal framework that integrates visual and textual features to predict the GIF sentiment. It also incorporates attributes including face emotion detection and OCR generated captions to capture the semantic aspects of the GIF. The developed classifier achieves an accuracy of 82.7% on Twitter GIFs, which is an improvement over state-of-the-art models. Moreover, we have based our research on the ReactionGIF dataset, analysing the variance in sentiment perceived by the author and sentiment induced in the reader

Related papers

• FACE-net: Factual Calibration and Emotion Augmentation for Retrieval-enhanced Emotional Video Captioning [81.33341786837974]
  Emotional Video Captioning (EVC) is an emerging task, which aims to describe factual content with the intrinsic emotions expressed in videos.<n>We propose a retrieval-enhanced framework with FActual and Emotion augmentation (FACE-net)<n>FACE-net collaboratively mines factual-emotional semantics and provides adaptive and accurate guidance for generation.
  arXiv  Detail & Related papers  (2026-03-18T07:53:15Z)
• Towards Deeper Emotional Reflection: Crafting Affective Image Filters with Generative Priors [60.113589550498176]
  Social media platforms enable users to express emotions by posting text with accompanying images.<n>We propose the Affective Image Filter (AIF) task, which aims to reflect visually-abstract emotions from text into visually-concrete images.
  arXiv  Detail & Related papers  (2025-12-19T09:24:22Z)
• Bridging Cognition and Emotion: Empathy-Driven Multimodal Misinformation Detection [56.644686934050576]
  Social media has become a major conduit for information dissemination, yet it also facilitates the rapid spread of misinformation. Traditional misinformation detection methods primarily focus on surface-level features, overlooking the crucial roles of human empathy in the propagation process. We propose the Dual-Aspect Empathy Framework (DAE), which integrates cognitive and emotional empathy to analyze misinformation from both the creator and reader perspectives.
  arXiv  Detail & Related papers  (2025-04-24T07:48:26Z)
• SentiFormer: Metadata Enhanced Transformer for Image Sentiment Analysis [8.823200970018487]
  We propose a novel Metadata Enhanced Transformer for sentiment analysis (SentiFormer) To adaptively learn the appropriate weights for each metadata, we then design an adaptive relevance learning module. We develop a cross-modal fusion module to fuse the adaptively learned representations and make the final prediction.
  arXiv  Detail & Related papers  (2025-02-21T09:22:23Z)
• Enriching Multimodal Sentiment Analysis through Textual Emotional Descriptions of Visual-Audio Content [56.62027582702816]
  Multimodal Sentiment Analysis seeks to unravel human emotions by amalgamating text, audio, and visual data. Yet, discerning subtle emotional nuances within audio and video expressions poses a formidable challenge. We introduce DEVA, a progressive fusion framework founded on textual sentiment descriptions.
  arXiv  Detail & Related papers  (2024-12-12T11:30:41Z)
• Target-Dependent Multimodal Sentiment Analysis Via Employing Visual-to Emotional-Caption Translation Network using Visual-Caption Pairs [13.922091192207718]
  This study presents a novel approach called the Visual-to-Emotional-Caption Translation Network (VECTN) technique. The primary objective of this strategy is to effectively acquire visual sentiment clues by analysing facial expressions. It effectively aligns and blends the obtained emotional clues with the target attribute of the caption mode. The experimental results show that the suggested model achieves an accuracy of 81.23% and a macro-F1 of 80.61% on the Twitter-15 dataset.
  arXiv  Detail & Related papers  (2024-08-05T15:56:55Z)
• StyleEDL: Style-Guided High-order Attention Network for Image Emotion Distribution Learning [69.06749934902464]
  We propose a style-guided high-order attention network for image emotion distribution learning termed StyleEDL. StyleEDL interactively learns stylistic-aware representations of images by exploring the hierarchical stylistic information of visual contents. In addition, we introduce a stylistic graph convolutional network to dynamically generate the content-dependent emotion representations.
  arXiv  Detail & Related papers  (2023-08-06T03:22:46Z)
• AoM: Detecting Aspect-oriented Information for Multimodal Aspect-Based Sentiment Analysis [14.173085246365826]
  This paper proposes an Aspect-oriented Method (AoM) to detect aspect-relevant semantic and sentiment information. To accurately aggregate sentiment information, we explicitly introduce sentiment embedding into AoM, and use a graph convolutional network to model the vision-text and text-text interaction.
  arXiv  Detail & Related papers  (2023-05-31T11:50:43Z)
• EDSA-Ensemble: an Event Detection Sentiment Analysis Ensemble Architecture [63.85863519876587]
  Using Sentiment Analysis to understand the polarity of each message belonging to an event, as well as the entire event, can help to better understand the general and individual feelings of significant trends and the dynamics on online social networks. We propose a new ensemble architecture, EDSA-Ensemble, that uses Event Detection and Sentiment Analysis to improve the detection of the polarity for current events from Social Media.
  arXiv  Detail & Related papers  (2023-01-30T11:56:08Z)
• Seeking Subjectivity in Visual Emotion Distribution Learning [93.96205258496697]
  Visual Emotion Analysis (VEA) aims to predict people's emotions towards different visual stimuli. Existing methods often predict visual emotion distribution in a unified network, neglecting the inherent subjectivity in its crowd voting process. We propose a novel textitSubjectivity Appraise-and-Match Network (SAMNet) to investigate the subjectivity in visual emotion distribution.
  arXiv  Detail & Related papers  (2022-07-25T02:20:03Z)
• Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resources [70.68526820807402]
  A real image is re-purposed to support other narratives by misrepresenting its context and/or elements. Our goal is an inspectable method that automates this time-consuming and reasoning-intensive process by fact-checking the image-context pairing. Our work offers the first step and benchmark for open-domain, content-based, multi-modal fact-checking.
  arXiv  Detail & Related papers  (2021-11-30T19:36:20Z)
• SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network [83.27291945217424]
  We propose a novel Scene-Object interreLated Visual Emotion Reasoning network (SOLVER) to predict emotions from images. To mine the emotional relationships between distinct objects, we first build up an Emotion Graph based on semantic concepts and visual features. We also design a Scene-Object Fusion Module to integrate scenes and objects, which exploits scene features to guide the fusion process of object features with the proposed scene-based attention mechanism.
  arXiv  Detail & Related papers  (2021-10-24T02:41:41Z)
• MONITOR: A Multimodal Fusion Framework to Assess Message Veracity in Social Networks [0.0]
  Users of social networks tend to post and share content with little restraint. Rumors and fake news can quickly spread on a huge scale. This may pose a threat to the credibility of social media and can cause serious consequences in real life.
  arXiv  Detail & Related papers  (2021-09-06T07:41:21Z)
• Visual Sentiment Analysis from Disaster Images in Social Media [11.075683976162766]
  This article focuses on visual sentiment analysis in a societal important domain, namely disaster analysis in social media. We propose a deep visual sentiment analyzer for disaster related images, covering different aspects of visual sentiment analysis. We believe the proposed system can contribute toward more livable communities by helping different stakeholders.
  arXiv  Detail & Related papers  (2020-09-04T11:29:52Z)
• Tweets Sentiment Analysis via Word Embeddings and Machine Learning Techniques [1.345251051985899]
  This paper aims to perform sentiment analysis of real-time 2019 election twitter data using the feature selection model word2vec and the machine learning algorithm random forest for sentiment classification. Word2vec improves the quality of features by considering contextual semantics of words in a text hence improving the accuracy of machine learning and sentiment analysis.
  arXiv  Detail & Related papers  (2020-07-05T08:10:30Z)
• EmotiCon: Context-Aware Multimodal Emotion Recognition using Frege's Principle [71.47160118286226]
  We present EmotiCon, a learning-based algorithm for context-aware perceived human emotion recognition from videos and images. Motivated by Frege's Context Principle from psychology, our approach combines three interpretations of context for emotion recognition. We report an Average Precision (AP) score of 35.48 across 26 classes, which is an improvement of 7-8 over prior methods.
  arXiv  Detail & Related papers  (2020-03-14T19:55:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.