IITK at SemEval-2024 Task 4: Hierarchical Embeddings for Detection of Persuasion Techniques in Memes

• URL: http://arxiv.org/abs/2404.04520v1
• Date: Sat, 6 Apr 2024 06:28:02 GMT
• Title: IITK at SemEval-2024 Task 4: Hierarchical Embeddings for Detection of Persuasion Techniques in Memes
• Authors: Shreenaga Chikoti, Shrey Mehta, Ashutosh Modi,
• Abstract summary: This paper proposes an ensemble of Class Definition Prediction (CDP) and hyperbolic embeddings-based approaches for this task. We enhance meme classification accuracy and comprehensiveness by integrating HypEmo's hierarchical label embeddings and a multi-task learning framework for emotion prediction.
• Score: 4.679320772294786
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Memes are one of the most popular types of content used in an online disinformation campaign. They are primarily effective on social media platforms since they can easily reach many users. Memes in a disinformation campaign achieve their goal of influencing the users through several rhetorical and psychological techniques, such as causal oversimplification, name-calling, and smear. The SemEval 2024 Task 4 \textit{Multilingual Detection of Persuasion Technique in Memes} on identifying such techniques in the memes is divided across three sub-tasks: ($\mathbf{1}$) Hierarchical multi-label classification using only textual content of the meme, ($\mathbf{2}$) Hierarchical multi-label classification using both, textual and visual content of the meme and ($\mathbf{3}$) Binary classification of whether the meme contains a persuasion technique or not using it's textual and visual content. This paper proposes an ensemble of Class Definition Prediction (CDP) and hyperbolic embeddings-based approaches for this task. We enhance meme classification accuracy and comprehensiveness by integrating HypEmo's hierarchical label embeddings (Chen et al., 2023) and a multi-task learning framework for emotion prediction. We achieve a hierarchical F1-score of 0.60, 0.67, and 0.48 on the respective sub-tasks.

Related papers

• Decoding Memes: A Comparative Study of Machine Learning Models for Template Identification [0.0]
  "meme template" is a layout or format that is used to create memes. Despite extensive research on meme virality, the task of automatically identifying meme templates remains a challenge. This paper presents a comprehensive comparison and evaluation of existing meme template identification methods.
  arXiv  Detail & Related papers  (2024-08-15T12:52:06Z)
• XMeCap: Meme Caption Generation with Sub-Image Adaptability [53.2509590113364]
  Humor, deeply rooted in societal meanings and cultural details, poses a unique challenge for machines. We introduce the textscXMeCap framework, which adopts supervised fine-tuning and reinforcement learning. textscXMeCap achieves an average evaluation score of 75.85 for single-image memes and 66.32 for multi-image memes, outperforming the best baseline by 3.71% and 4.82%, respectively.
  arXiv  Detail & Related papers  (2024-07-24T10:51:46Z)
• BCAmirs at SemEval-2024 Task 4: Beyond Words: A Multimodal and Multilingual Exploration of Persuasion in Memes [17.09830912625338]
  We introduce a caption generation step to assess the modality gap and the impact of additional semantic information from images. Our best model utilizes GPT-4 generated captions alongside meme text to fine-tune RoBERTa as the text encoder and CLIP as the image encoder.
  arXiv  Detail & Related papers  (2024-04-03T19:17:43Z)
• Meme-ingful Analysis: Enhanced Understanding of Cyberbullying in Memes Through Multimodal Explanations [48.82168723932981]
  We introduce em MultiBully-Ex, the first benchmark dataset for multimodal explanation from code-mixed cyberbullying memes. A Contrastive Language-Image Pretraining (CLIP) approach has been proposed for visual and textual explanation of a meme.
  arXiv  Detail & Related papers  (2024-01-18T11:24:30Z)
• A Template Is All You Meme [83.05919383106715]
  We release a knowledge base of memes and information found on www.knowyourmeme.com, composed of more than 54,000 images. We hypothesize that meme templates can be used to inject models with the context missing from previous approaches.
  arXiv  Detail & Related papers  (2023-11-11T19:38:14Z)
• HuBERTopic: Enhancing Semantic Representation of HuBERT through Self-supervision Utilizing Topic Model [62.995175485416]
  We propose a new approach to enrich the semantic representation of HuBERT. An auxiliary topic classification task is added to HuBERT by using topic labels as teachers. Experimental results demonstrate that our method achieves comparable or better performance than the baseline in most tasks.
  arXiv  Detail & Related papers  (2023-10-06T02:19:09Z)
• MEMEX: Detecting Explanatory Evidence for Memes via Knowledge-Enriched Contextualization [31.209594252045566]
  We propose a novel task, MEMEX, given a meme and a related document, the aim is to mine the context that succinctly explains the background of the meme. To benchmark MCC, we propose MIME, a multimodal neural framework that uses common sense enriched meme representation and a layered approach to capture the cross-modal semantic dependencies between the meme and the context.
  arXiv  Detail & Related papers  (2023-05-25T10:19:35Z)
• What do you MEME? Generating Explanations for Visual Semantic Role Labelling in Memes [42.357272117919464]
  We introduce a novel task - EXCLAIM, generating explanations for visual semantic role labeling in memes. To this end, we curate ExHVV, a novel dataset that offers natural language explanations of connotative roles for three types of entities. We also posit LUMEN, a novel multimodal, multi-task learning framework that endeavors to address EXCLAIM optimally.
  arXiv  Detail & Related papers  (2022-12-01T18:21:36Z)
• Detecting and Understanding Harmful Memes: A Survey [48.135415967633676]
  We offer a comprehensive survey with a focus on harmful memes. One interesting finding is that many types of harmful memes are not really studied, e.g., such featuring self-harm and extremism. Another observation is that memes can propagate globally through repackaging in different languages and that they can also be multilingual.
  arXiv  Detail & Related papers  (2022-05-09T13:43:27Z)
• Volta at SemEval-2021 Task 6: Towards Detecting Persuasive Texts and Images using Textual and Multimodal Ensemble [7.817598216459955]
  We propose a transfer learning approach to fine-tune BERT-based models in different modalities. We achieve an F1-score of 57.0, 48.2, and 52.1 in the corresponding subtasks.
  arXiv  Detail & Related papers  (2021-06-01T05:41:03Z)
• MATCH: Metadata-Aware Text Classification in A Large Hierarchy [60.59183151617578]
  MATCH is an end-to-end framework that leverages both metadata and hierarchy information. We propose different ways to regularize the parameters and output probability of each child label by its parents. Experiments on two massive text datasets with large-scale label hierarchies demonstrate the effectiveness of MATCH.
  arXiv  Detail & Related papers  (2021-02-15T05:23:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.