Exercise? I thought you said 'Extra Fries': Leveraging Sentence Demarcations and Multi-hop Attention for Meme Affect Analysis

• URL: http://arxiv.org/abs/2103.12377v1
• Date: Tue, 23 Mar 2021 08:21:37 GMT
• Title: Exercise? I thought you said 'Extra Fries': Leveraging Sentence Demarcations and Multi-hop Attention for Meme Affect Analysis
• Authors: Shraman Pramanick, Md Shad Akhtar, Tanmoy Chakraborty
• Abstract summary: We propose a multi-hop attention-based deep neural network framework, called MHA-MEME. Its prime objective is to leverage the spatial-domain correspondence between the visual modality (an image) and various textual segments to extract fine-grained feature representations for classification. We evaluate MHA-MEME on the 'Memotion Analysis' dataset for all three sub-tasks - sentiment classification, affect classification, and affect class quantification.
• Score: 18.23523076710257
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Today's Internet is awash in memes as they are humorous, satirical, or ironic which make people laugh. According to a survey, 33% of social media users in age bracket [13-35] send memes every day, whereas more than 50% send every week. Some of these memes spread rapidly within a very short time-frame, and their virality depends on the novelty of their (textual and visual) content. A few of them convey positive messages, such as funny or motivational quotes; while others are meant to mock/hurt someone's feelings through sarcastic or offensive messages. Despite the appealing nature of memes and their rapid emergence on social media, effective analysis of memes has not been adequately attempted to the extent it deserves. In this paper, we attempt to solve the same set of tasks suggested in the SemEval'20-Memotion Analysis competition. We propose a multi-hop attention-based deep neural network framework, called MHA-MEME, whose prime objective is to leverage the spatial-domain correspondence between the visual modality (an image) and various textual segments to extract fine-grained feature representations for classification. We evaluate MHA-MEME on the 'Memotion Analysis' dataset for all three sub-tasks - sentiment classification, affect classification, and affect class quantification. Our comparative study shows sota performances of MHA-MEME for all three tasks compared to the top systems that participated in the competition. Unlike all the baselines which perform inconsistently across all three tasks, MHA-MEME outperforms baselines in all the tasks on average. Moreover, we validate the generalization of MHA-MEME on another set of manually annotated test samples and observe it to be consistent. Finally, we establish the interpretability of MHA-MEME.

Related papers

• XMeCap: Meme Caption Generation with Sub-Image Adaptability [53.2509590113364]
  Humor, deeply rooted in societal meanings and cultural details, poses a unique challenge for machines. We introduce the textscXMeCap framework, which adopts supervised fine-tuning and reinforcement learning. textscXMeCap achieves an average evaluation score of 75.85 for single-image memes and 66.32 for multi-image memes, outperforming the best baseline by 3.71% and 4.82%, respectively.
  arXiv  Detail & Related papers  (2024-07-24T10:51:46Z)
• IITK at SemEval-2024 Task 4: Hierarchical Embeddings for Detection of Persuasion Techniques in Memes [4.679320772294786]
  This paper proposes an ensemble of Class Definition Prediction (CDP) and hyperbolic embeddings-based approaches for this task. We enhance meme classification accuracy and comprehensiveness by integrating HypEmo's hierarchical label embeddings and a multi-task learning framework for emotion prediction.
  arXiv  Detail & Related papers  (2024-04-06T06:28:02Z)
• Overview of Memotion 3: Sentiment and Emotion Analysis of Codemixed Hinglish Memes [36.34201719103715]
  We present the overview of the Memotion 3 shared task, as part of the DeFactify 2 workshop at AAAI-23. The task released an annotated dataset of Hindi-English code-mixed memes based on their Sentiment (Task A), Emotion (Task B), and Emotion intensity (Task C) Over 50 teams registered for the shared task and 5 made final submissions to the test set of the Memotion 3 dataset.
  arXiv  Detail & Related papers  (2023-09-12T18:47:29Z)
• Incorporating Emotions into Health Mention Classification Task on Social Media [70.23889100356091]
  We present a framework for health mention classification that incorporates affective features. We evaluate our approach on 5 HMC-related datasets from different social media platforms. Our results indicate that HMC models infused with emotional knowledge are an effective alternative.
  arXiv  Detail & Related papers  (2022-12-09T18:38:41Z)
• What do you MEME? Generating Explanations for Visual Semantic Role Labelling in Memes [42.357272117919464]
  We introduce a novel task - EXCLAIM, generating explanations for visual semantic role labeling in memes. To this end, we curate ExHVV, a novel dataset that offers natural language explanations of connotative roles for three types of entities. We also posit LUMEN, a novel multimodal, multi-task learning framework that endeavors to address EXCLAIM optimally.
  arXiv  Detail & Related papers  (2022-12-01T18:21:36Z)
• Seeking Subjectivity in Visual Emotion Distribution Learning [93.96205258496697]
  Visual Emotion Analysis (VEA) aims to predict people's emotions towards different visual stimuli. Existing methods often predict visual emotion distribution in a unified network, neglecting the inherent subjectivity in its crowd voting process. We propose a novel textitSubjectivity Appraise-and-Match Network (SAMNet) to investigate the subjectivity in visual emotion distribution.
  arXiv  Detail & Related papers  (2022-07-25T02:20:03Z)
• DISARM: Detecting the Victims Targeted by Harmful Memes [49.12165815990115]
  DISARM is a framework that uses named entity recognition and person identification to detect harmful memes. We show that DISARM significantly outperforms ten unimodal and multimodal systems. It can reduce the relative error rate for harmful target identification by up to 9 points absolute over several strong multimodal rivals.
  arXiv  Detail & Related papers  (2022-05-11T19:14:26Z)
• Detecting and Understanding Harmful Memes: A Survey [48.135415967633676]
  We offer a comprehensive survey with a focus on harmful memes. One interesting finding is that many types of harmful memes are not really studied, e.g., such featuring self-harm and extremism. Another observation is that memes can propagate globally through repackaging in different languages and that they can also be multilingual.
  arXiv  Detail & Related papers  (2022-05-09T13:43:27Z)
• SemEval-2020 Task 8: Memotion Analysis -- The Visuo-Lingual Metaphor! [20.55903557920223]
  The objective of this proposal is to bring the attention of the research community towards the automatic processing of Internet memes. The task Memotion analysis released approx 10K annotated memes, with human-annotated labels namely sentiment (positive, negative, neutral), type of emotion (sarcastic, funny, offensive, motivation) and corresponding intensity. The challenge consisted of three subtasks: sentiment (positive, negative, and neutral) analysis of memes, overall emotion (humour, sarcasm, offensive, and motivational) classification of memes, and classifying intensity of meme emotion.
  arXiv  Detail & Related papers  (2020-08-09T18:17:33Z)
• IITK at SemEval-2020 Task 8: Unimodal and Bimodal Sentiment Analysis of Internet Memes [2.2385755093672044]
  We present our approaches for the Memotion Analysis problem as posed in SemEval-2020 Task 8. The goal of this task is to classify memes based on their emotional content and sentiment. Our results show that a text-only approach, a simple Feed Forward Neural Network (FFNN) with Word2vec embeddings as input, performs superior to all the others.
  arXiv  Detail & Related papers  (2020-07-21T14:06:26Z)
• EmotiCon: Context-Aware Multimodal Emotion Recognition using Frege's Principle [71.47160118286226]
  We present EmotiCon, a learning-based algorithm for context-aware perceived human emotion recognition from videos and images. Motivated by Frege's Context Principle from psychology, our approach combines three interpretations of context for emotion recognition. We report an Average Precision (AP) score of 35.48 across 26 classes, which is an improvement of 7-8 over prior methods.
  arXiv  Detail & Related papers  (2020-03-14T19:55:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.