An AutoML-based Approach to Multimodal Image Sentiment Analysis
- URL: http://arxiv.org/abs/2102.08092v1
- Date: Tue, 16 Feb 2021 11:28:50 GMT
- Title: An AutoML-based Approach to Multimodal Image Sentiment Analysis
- Authors: Vasco Lopes, Ant\'onio Gaspar, Lu\'is A. Alexandre, Jo\~ao Cordeiro
- Abstract summary: We propose a method that combines both textual and image individual sentiment analysis into a final fused classification based on AutoML.
Our method achieved state-of-the-art performance in the B-T4SA dataset, with 95.19% accuracy.
- Score: 1.0499611180329804
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Sentiment analysis is a research topic focused on analysing data to extract
information related to the sentiment that it causes. Applications of sentiment
analysis are wide, ranging from recommendation systems, and marketing to
customer satisfaction. Recent approaches evaluate textual content using Machine
Learning techniques that are trained over large corpora. However, as social
media grown, other data types emerged in large quantities, such as images.
Sentiment analysis in images has shown to be a valuable complement to textual
data since it enables the inference of the underlying message polarity by
creating context and connections. Multimodal sentiment analysis approaches
intend to leverage information of both textual and image content to perform an
evaluation. Despite recent advances, current solutions still flounder in
combining both image and textual information to classify social media data,
mainly due to subjectivity, inter-class homogeneity and fusion data
differences. In this paper, we propose a method that combines both textual and
image individual sentiment analysis into a final fused classification based on
AutoML, that performs a random search to find the best model. Our method
achieved state-of-the-art performance in the B-T4SA dataset, with 95.19%
accuracy.
Related papers
- WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual
World Knowledge [73.76722241704488]
We propose a plug-in framework named WisdoM to leverage the contextual world knowledge induced from the large vision-language models (LVLMs) for enhanced multimodal sentiment analysis.
We show that our approach has substantial improvements over several state-of-the-art methods.
arXiv Detail & Related papers (2024-01-12T16:08:07Z) - From Text to Pixels: A Context-Aware Semantic Synergy Solution for
Infrared and Visible Image Fusion [66.33467192279514]
We introduce a text-guided multi-modality image fusion method that leverages the high-level semantics from textual descriptions to integrate semantics from infrared and visible images.
Our method not only produces visually superior fusion results but also achieves a higher detection mAP over existing methods, achieving state-of-the-art results.
arXiv Detail & Related papers (2023-12-31T08:13:47Z) - Harnessing the Power of Text-image Contrastive Models for Automatic
Detection of Online Misinformation [50.46219766161111]
We develop a self-learning model to explore the constrastive learning in the domain of misinformation identification.
Our model shows the superior performance of non-matched image-text pair detection when the training data is insufficient.
arXiv Detail & Related papers (2023-04-19T02:53:59Z) - Holistic Visual-Textual Sentiment Analysis with Prior Models [64.48229009396186]
We propose a holistic method that achieves robust visual-textual sentiment analysis.
The proposed method consists of four parts: (1) a visual-textual branch to learn features directly from data for sentiment analysis, (2) a visual expert branch with a set of pre-trained "expert" encoders to extract selected semantic visual features, (3) a CLIP branch to implicitly model visual-textual correspondence, and (4) a multimodal feature fusion network based on BERT to fuse multimodal features and make sentiment predictions.
arXiv Detail & Related papers (2022-11-23T14:40:51Z) - Transfer Learning with Joint Fine-Tuning for Multimodal Sentiment
Analysis [0.6091702876917281]
We introduce a transfer learning approach using joint fine-tuning for sentiment analysis.
Our proposal allows flexibility when incorporating any pre-trained model for texts and images during the joint fine-tuning stage.
arXiv Detail & Related papers (2022-10-11T21:16:14Z) - A Comprehensive Review of Visual-Textual Sentiment Analysis from Social
Media Networks [2.048226951354646]
Social media networks have become a significant aspect of people's lives, serving as a platform for their ideas, opinions and emotions.
The analysis of these feelings revealed various applications, including brand evaluations, YouTube film reviews and healthcare applications.
Our study focuses on the field of multimodal sentiment analysis, which examines visual and textual data posted on social media networks.
arXiv Detail & Related papers (2022-07-05T16:28:47Z) - Where Does the Performance Improvement Come From? - A Reproducibility
Concern about Image-Text Retrieval [85.03655458677295]
Image-text retrieval has gradually become a major research direction in the field of information retrieval.
We first examine the related concerns and why the focus is on image-text retrieval tasks.
We analyze various aspects of the reproduction of pretrained and nonpretrained retrieval models.
arXiv Detail & Related papers (2022-03-08T05:01:43Z) - A Novel Context-Aware Multimodal Framework for Persian Sentiment
Analysis [19.783517380422854]
We present a first of its kind Persian multimodal dataset comprising more than 800 utterances.
We present a novel context-aware multimodal sentiment analysis framework.
We employ both decision-level (late) and feature-level (early) fusion methods to integrate affective cross-modal information.
arXiv Detail & Related papers (2021-03-03T19:09:01Z) - Weakly-Supervised Aspect-Based Sentiment Analysis via Joint
Aspect-Sentiment Topic Embedding [71.2260967797055]
We propose a weakly-supervised approach for aspect-based sentiment analysis.
We learn sentiment, aspect> joint topic embeddings in the word embedding space.
We then use neural models to generalize the word-level discriminative information.
arXiv Detail & Related papers (2020-10-13T21:33:24Z) - Visual Sentiment Analysis from Disaster Images in Social Media [11.075683976162766]
This article focuses on visual sentiment analysis in a societal important domain, namely disaster analysis in social media.
We propose a deep visual sentiment analyzer for disaster related images, covering different aspects of visual sentiment analysis.
We believe the proposed system can contribute toward more livable communities by helping different stakeholders.
arXiv Detail & Related papers (2020-09-04T11:29:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.