TIB-VA at SemEval-2022 Task 5: A Multimodal Architecture for the Detection and Classification of Misogynous Memes

• URL: http://arxiv.org/abs/2204.06299v1
• Date: Wed, 13 Apr 2022 11:03:21 GMT
• Title: TIB-VA at SemEval-2022 Task 5: A Multimodal Architecture for the Detection and Classification of Misogynous Memes
• Authors: Sherzod Hakimov and Gullal S. Cheema and Ralph Ewerth
• Abstract summary: We present a multimodal architecture that combines textual and visual features in order to detect misogynous meme content. Our solution obtained the best result in the Task-B where the challenge is to classify whether a given document is misogynous.
• Score: 9.66022279280394
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The detection of offensive, hateful content on social media is a challenging problem that affects many online users on a daily basis. Hateful content is often used to target a group of people based on ethnicity, gender, religion and other factors. The hate or contempt toward women has been increasing on social platforms. Misogynous content detection is especially challenging when textual and visual modalities are combined to form a single context, e.g., an overlay text embedded on top of an image, also known as meme. In this paper, we present a multimodal architecture that combines textual and visual features in order to detect misogynous meme content. The proposed architecture is evaluated in the SemEval-2022 Task 5: MAMI - Multimedia Automatic Misogyny Identification challenge under the team name TIB-VA. Our solution obtained the best result in the Task-B where the challenge is to classify whether a given document is misogynous and further identify the main sub-classes of shaming, stereotype, objectification, and violence.

Related papers

• A multitask learning framework for leveraging subjectivity of annotators to identify misogyny [47.175010006458436]
  We propose a multitask learning approach to enhance the performance of the misogyny identification systems. We incorporated diverse perspectives from annotators in our model design, considering gender and age across six profile groups. This research advances content moderation and highlights the importance of embracing diverse perspectives to build effective online moderation systems.
  arXiv  Detail & Related papers  (2024-06-22T15:06:08Z)
• SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Models [63.946809247201905]
  We introduce a new benchmark, namely SHIELD, to evaluate the ability of MLLMs on face spoofing and forgery detection. We design true/false and multiple-choice questions to evaluate multimodal face data in these two face security tasks. The results indicate that MLLMs hold substantial potential in the face security domain.
  arXiv  Detail & Related papers  (2024-02-06T17:31:36Z)
• GPT-4V(ision) as A Social Media Analysis Engine [77.23394183063238]
  This paper explores GPT-4V's capabilities for social multimedia analysis. We select five representative tasks, including sentiment analysis, hate speech detection, fake news identification, demographic inference, and political ideology detection. GPT-4V demonstrates remarkable efficacy in these tasks, showcasing strengths such as joint understanding of image-text pairs, contextual and cultural awareness, and extensive commonsense knowledge.
  arXiv  Detail & Related papers  (2023-11-13T18:36:50Z)
• Countering Malicious Content Moderation Evasion in Online Social Networks: Simulation and Detection of Word Camouflage [64.78260098263489]
  Twisting and camouflaging keywords are among the most used techniques to evade platform content moderation systems. This article contributes significantly to countering malicious information by developing multilingual tools to simulate and detect new methods of evasion of content.
  arXiv  Detail & Related papers  (2022-12-27T16:08:49Z)
• Deep Multi-Task Models for Misogyny Identification and Categorization on Arabic Social Media [6.6410040715586005]
  In this paper, we present the submitted systems to the first Arabic Misogyny Identification shared task. We investigate three multi-task learning models as well as their single-task counterparts. In order to encode the input text, our models rely on the pre-trained MARBERT language model.
  arXiv  Detail & Related papers  (2022-06-16T18:54:37Z)
• UPB at SemEval-2022 Task 5: Enhancing UNITER with Image Sentiment and Graph Convolutional Networks for Multimedia Automatic Misogyny Identification [0.3437656066916039]
  We describe our classification systems submitted to the SemEval-2022 Task 5: MAMI - Multimedia Automatic Misogyny Identification. Our best model reaches an F1-score of 71.4% in Sub-task A and 67.3% for Sub-task B positioning our team in the upper third of the leaderboard.
  arXiv  Detail & Related papers  (2022-05-29T21:12:36Z)
• DISARM: Detecting the Victims Targeted by Harmful Memes [49.12165815990115]
  DISARM is a framework that uses named entity recognition and person identification to detect harmful memes. We show that DISARM significantly outperforms ten unimodal and multimodal systems. It can reduce the relative error rate for harmful target identification by up to 9 points absolute over several strong multimodal rivals.
  arXiv  Detail & Related papers  (2022-05-11T19:14:26Z)
• Detecting and Understanding Harmful Memes: A Survey [48.135415967633676]
  We offer a comprehensive survey with a focus on harmful memes. One interesting finding is that many types of harmful memes are not really studied, e.g., such featuring self-harm and extremism. Another observation is that memes can propagate globally through repackaging in different languages and that they can also be multilingual.
  arXiv  Detail & Related papers  (2022-05-09T13:43:27Z)
• RubCSG at SemEval-2022 Task 5: Ensemble learning for identifying misogynous MEMEs [12.979213013465882]
  This work presents an ensemble system based on various uni-modal and bi-modal model architectures developed for the SemEval 2022 Task 5: MAMI-Multimedia Automatic Misogyny Identification.
  arXiv  Detail & Related papers  (2022-04-08T09:27:28Z)
• AMS_ADRN at SemEval-2022 Task 5: A Suitable Image-text Multimodal Joint Modeling Method for Multi-task Misogyny Identification [3.5382535469099436]
  Women are influential online, especially in image-based social media such as Twitter and Instagram. In this paper, we describe the system developed by our team for SemEval-2022 Task 5: Multimedia Automatic Misogyny Identification.
  arXiv  Detail & Related papers  (2022-02-18T09:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.