DISARM: Detecting the Victims Targeted by Harmful Memes

• URL: http://arxiv.org/abs/2205.05738v1
• Date: Wed, 11 May 2022 19:14:26 GMT
• Title: DISARM: Detecting the Victims Targeted by Harmful Memes
• Authors: Shivam Sharma, Md. Shad Akhtar, Preslav Nakov, Tanmoy Chakraborty
• Abstract summary: DISARM is a framework that uses named entity recognition and person identification to detect harmful memes. We show that DISARM significantly outperforms ten unimodal and multimodal systems. It can reduce the relative error rate for harmful target identification by up to 9 points absolute over several strong multimodal rivals.
• Score: 49.12165815990115
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Internet memes have emerged as an increasingly popular means of communication on the Web. Although typically intended to elicit humour, they have been increasingly used to spread hatred, trolling, and cyberbullying, as well as to target specific individuals, communities, or society on political, socio-cultural, and psychological grounds. While previous work has focused on detecting harmful, hateful, and offensive memes, identifying whom they attack remains a challenging and underexplored area. Here we aim to bridge this gap. In particular, we create a dataset where we annotate each meme with its victim(s) such as the name of the targeted person(s), organization(s), and community(ies). We then propose DISARM (Detecting vIctimS targeted by hARmful Memes), a framework that uses named entity recognition and person identification to detect all entities a meme is referring to, and then, incorporates a novel contextualized multimodal deep neural network to classify whether the meme intends to harm these entities. We perform several systematic experiments on three test setups, corresponding to entities that are (a) all seen while training, (b) not seen as a harmful target on training, and (c) not seen at all on training. The evaluation results show that DISARM significantly outperforms ten unimodal and multimodal systems. Finally, we show that DISARM is interpretable and comparatively more generalizable and that it can reduce the relative error rate for harmful target identification by up to 9 points absolute over several strong multimodal rivals.

Related papers

• Deciphering Hate: Identifying Hateful Memes and Their Targets [4.574830585715128]
  We introduce a novel dataset for detecting hateful memes in Bengali, BHM (Bengali Hateful Memes) The dataset consists of 7,148 memes with Bengali as well as code-mixed captions, tailored for two tasks: (i) detecting hateful memes, and (ii) detecting the social entities they target. To solve these tasks, we propose DORA, a multimodal deep neural network that systematically extracts the significant modality features from the memes.
  arXiv  Detail & Related papers  (2024-03-16T06:39:41Z)
• Zero shot VLMs for hate meme detection: Are we there yet? [9.970031080934003]
  This study investigates the efficacy of visual language models in handling intricate tasks such as hate meme detection. We observe that large VLMs are still vulnerable for zero-shot hate meme detection.
  arXiv  Detail & Related papers  (2024-02-19T15:03:04Z)
• Meme-ingful Analysis: Enhanced Understanding of Cyberbullying in Memes Through Multimodal Explanations [48.82168723932981]
  We introduce em MultiBully-Ex, the first benchmark dataset for multimodal explanation from code-mixed cyberbullying memes. A Contrastive Language-Image Pretraining (CLIP) approach has been proposed for visual and textual explanation of a meme.
  arXiv  Detail & Related papers  (2024-01-18T11:24:30Z)
• Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim? [39.55435707149863]
  We aim to understand whether the meme glorifies, vilifies, or victimizes each entity it refers to. Our proposed model achieves an improvement of 4% over the best baseline and 1% over the best competing stand-alone submission.
  arXiv  Detail & Related papers  (2023-01-26T16:55:15Z)
• DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation [72.18912216025029]
  We present DisinfoMeme to help detect disinformation memes. The dataset contains memes mined from Reddit covering three current topics: the COVID-19 pandemic, the Black Lives Matter movement, and veganism/vegetarianism.
  arXiv  Detail & Related papers  (2022-05-25T09:54:59Z)
• Detecting and Understanding Harmful Memes: A Survey [48.135415967633676]
  We offer a comprehensive survey with a focus on harmful memes. One interesting finding is that many types of harmful memes are not really studied, e.g., such featuring self-harm and extremism. Another observation is that memes can propagate globally through repackaging in different languages and that they can also be multilingual.
  arXiv  Detail & Related papers  (2022-05-09T13:43:27Z)
• Feels Bad Man: Dissecting Automated Hateful Meme Detection Through the Lens of Facebook's Challenge [10.775419935941008]
  We assess the efficacy of current state-of-the-art multimodal machine learning models toward hateful meme detection. We use two benchmark datasets comprising 12,140 and 10,567 images from 4chan's "Politically Incorrect" board (/pol/) and Facebook's Hateful Memes Challenge dataset. We conduct three experiments to determine the importance of multimodality on classification performance, the influential capacity of fringe Web communities on mainstream social platforms and vice versa.
  arXiv  Detail & Related papers  (2022-02-17T07:52:22Z)
• MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets [28.877314859737197]
  We aim to solve two novel tasks: detecting harmful memes and identifying the social entities they target. In particular, we aim to solve two novel tasks: detecting harmful memes and identifying the social entities they target. We propose MOMENTA, a novel multimodal (text + image) deep neural model, which uses global and local perspectives to detect harmful memes.
  arXiv  Detail & Related papers  (2021-09-11T04:29:32Z)
• Double Targeted Universal Adversarial Perturbations [83.60161052867534]
  We introduce a double targeted universal adversarial perturbations (DT-UAPs) to bridge the gap between the instance-discriminative image-dependent perturbations and the generic universal perturbations. We show the effectiveness of the proposed DTA algorithm on a wide range of datasets and also demonstrate its potential as a physical attack.
  arXiv  Detail & Related papers  (2020-10-07T09:08:51Z)
• Measurement-driven Security Analysis of Imperceptible Impersonation Attacks [54.727945432381716]
  We study the exploitability of Deep Neural Network-based Face Recognition systems. We show that factors such as skin color, gender, and age, impact the ability to carry out an attack on a specific target victim. We also study the feasibility of constructing universal attacks that are robust to different poses or views of the attacker's face.
  arXiv  Detail & Related papers  (2020-08-26T19:27:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.