Feels Bad Man: Dissecting Automated Hateful Meme Detection Through the Lens of Facebook's Challenge

• URL: http://arxiv.org/abs/2202.08492v1
• Date: Thu, 17 Feb 2022 07:52:22 GMT
• Title: Feels Bad Man: Dissecting Automated Hateful Meme Detection Through the Lens of Facebook's Challenge
• Authors: Catherine Jennifer, Fatemeh Tahmasbi, Jeremy Blackburn, Gianluca Stringhini, Savvas Zannettou, and Emiliano De Cristofaro
• Abstract summary: We assess the efficacy of current state-of-the-art multimodal machine learning models toward hateful meme detection. We use two benchmark datasets comprising 12,140 and 10,567 images from 4chan's "Politically Incorrect" board (/pol/) and Facebook's Hateful Memes Challenge dataset. We conduct three experiments to determine the importance of multimodality on classification performance, the influential capacity of fringe Web communities on mainstream social platforms and vice versa.
• Score: 10.775419935941008
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Internet memes have become a dominant method of communication; at the same time, however, they are also increasingly being used to advocate extremism and foster derogatory beliefs. Nonetheless, we do not have a firm understanding as to which perceptual aspects of memes cause this phenomenon. In this work, we assess the efficacy of current state-of-the-art multimodal machine learning models toward hateful meme detection, and in particular with respect to their generalizability across platforms. We use two benchmark datasets comprising 12,140 and 10,567 images from 4chan's "Politically Incorrect" board (/pol/) and Facebook's Hateful Memes Challenge dataset to train the competition's top-ranking machine learning models for the discovery of the most prominent features that distinguish viral hateful memes from benign ones. We conduct three experiments to determine the importance of multimodality on classification performance, the influential capacity of fringe Web communities on mainstream social platforms and vice versa, and the models' learning transferability on 4chan memes. Our experiments show that memes' image characteristics provide a greater wealth of information than its textual content. We also find that current systems developed for online detection of hate speech in memes necessitate further concentration on its visual elements to improve their interpretation of underlying cultural connotations, implying that multimodal models fail to adequately grasp the intricacies of hate speech in memes and generalize across social media platforms.

Related papers

• Meme-ingful Analysis: Enhanced Understanding of Cyberbullying in Memes Through Multimodal Explanations [48.82168723932981]
  We introduce em MultiBully-Ex, the first benchmark dataset for multimodal explanation from code-mixed cyberbullying memes. A Contrastive Language-Image Pretraining (CLIP) approach has been proposed for visual and textual explanation of a meme.
  arXiv  Detail & Related papers  (2024-01-18T11:24:30Z)
• Contextualizing Internet Memes Across Social Media Platforms [8.22187358555391]
  We investigate whether internet memes can be contextualized by using a semantic repository of knowledge, namely, a knowledge graph. We collect thousands of potential internet meme posts from two social media platforms, namely Reddit and Discord, and develop an extract-transform-load procedure to create a data lake with candidate meme posts. By using vision transformer-based similarity, we match these candidates against the memes cataloged in IMKG -- a recently released knowledge graph of internet memes.
  arXiv  Detail & Related papers  (2023-11-18T20:18:18Z)
• On the Evolution of (Hateful) Memes by Means of Multimodal Contrastive Learning [18.794226796466962]
  We study how hateful memes are created by combining visual elements from multiple images or fusing textual information with a hateful image. Using our framework on a dataset extracted from 4chan, we find 3.3K variants of the Happy Merchant meme. We envision that our framework can be used to aid human moderators by flagging new variants of hateful memes.
  arXiv  Detail & Related papers  (2022-12-13T13:38:04Z)
• DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation [72.18912216025029]
  We present DisinfoMeme to help detect disinformation memes. The dataset contains memes mined from Reddit covering three current topics: the COVID-19 pandemic, the Black Lives Matter movement, and veganism/vegetarianism.
  arXiv  Detail & Related papers  (2022-05-25T09:54:59Z)
• DISARM: Detecting the Victims Targeted by Harmful Memes [49.12165815990115]
  DISARM is a framework that uses named entity recognition and person identification to detect harmful memes. We show that DISARM significantly outperforms ten unimodal and multimodal systems. It can reduce the relative error rate for harmful target identification by up to 9 points absolute over several strong multimodal rivals.
  arXiv  Detail & Related papers  (2022-05-11T19:14:26Z)
• Detecting and Understanding Harmful Memes: A Survey [48.135415967633676]
  We offer a comprehensive survey with a focus on harmful memes. One interesting finding is that many types of harmful memes are not really studied, e.g., such featuring self-harm and extremism. Another observation is that memes can propagate globally through repackaging in different languages and that they can also be multilingual.
  arXiv  Detail & Related papers  (2022-05-09T13:43:27Z)
• Detecting Harmful Memes and Their Targets [27.25262711136056]
  We present HarMeme, the first benchmark dataset, containing 3,544 memes related to COVID-19. In the first stage, we labeled a meme as very harmful, partially harmful, or harmless; in the second stage, we further annotated the type of target(s) that each harmful meme points to. The evaluation results using ten unimodal and multimodal models highlight the importance of using multimodal signals for both tasks.
  arXiv  Detail & Related papers  (2021-09-24T17:11:42Z)
• Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset [47.65948529524281]
  We collect hateful and non-hateful memes from Pinterest to evaluate out-of-sample performance on models pre-trained on the Facebook dataset. We find that memes in the wild differ in two key aspects: 1) Captions must be extracted via OCR, and 2) Memes are more diverse than traditional memes', including screenshots of conversations or text on a plain background.
  arXiv  Detail & Related papers  (2021-07-09T09:04:05Z)
• Entropy and complexity unveil the landscape of memes evolution [105.59074436693487]
  We study the evolution of 2 million visual memes from Reddit over ten years, from 2011 to 2020. We find support for the hypothesis that memes are part of an emerging form of internet metalanguage.
  arXiv  Detail & Related papers  (2021-05-26T07:41:09Z)
• Multimodal Learning for Hateful Memes Detection [6.6881085567421605]
  We propose a novel method that incorporates the image captioning process into the memes detection process. Our model achieves promising results on the Hateful Memes Detection Challenge.
  arXiv  Detail & Related papers  (2020-11-25T16:49:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.