I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memes

• URL: http://arxiv.org/abs/2603.23229v1
• Date: Tue, 24 Mar 2026 14:02:48 GMT
• Title: I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memes
• Abstract summary: We evaluate eight state-of-the-art generative large language models (MLLMs) on their ability to detect and explain six types of figurative meaning.<n>Our findings indicate that all models exhibit a strong bias to associate a meme with figurative meaning, even when no such meaning is present.
• Score: 53.717874073948984
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Internet memes represent a popular form of multimodal online communication and often use figurative elements to convey layered meaning through the combination of text and images. However, it remains largely unclear how multimodal large language models (MLLMs) combine and interpret visual and textual information to identify figurative meaning in memes. To address this gap, we evaluate eight state-of-the-art generative MLLMs across three datasets on their ability to detect and explain six types of figurative meaning. In addition, we conduct a human evaluation of the explanations generated by these MLLMs, assessing whether the provided reasoning supports the predicted label and whether it remains faithful to the original meme content. Our findings indicate that all models exhibit a strong bias to associate a meme with figurative meaning, even when no such meaning is present. Qualitative analysis further shows that correct predictions are not always accompanied by faithful explanations.

Related papers

• Are Large Language Models Chronically Online Surfers? A Dataset for Chinese Internet Meme Explanation [4.797464143831952]
  Large language models (LLMs) are trained on vast amounts of text from the Internet, but do they truly understand the viral content that rapidly spreads online?<n>ChiME is a dataset for CHinese Internet Meme Explanation.<n>The dataset comprises popular phrase-based memes from the Chinese Internet, annotated with detailed information on their meaning, origin, example sentences, types, etc.
  arXiv  Detail & Related papers  (2025-10-01T06:41:46Z)
• MemeReaCon: Probing Contextual Meme Understanding in Large Vision-Language Models [50.2355423914562]
  We introduce MemeReaCon, a novel benchmark designed to evaluate how Large Vision Language Models (LVLMs) understand memes in their original context.<n>We collected memes from five different Reddit communities, keeping each meme's image, the post text, and user comments together.<n>Our tests with leading LVLMs show a clear weakness: models either fail to interpret critical information in the contexts, or overly focus on visual details while overlooking communicative purpose.
  arXiv  Detail & Related papers  (2025-05-23T03:27:23Z)
• XMeCap: Meme Caption Generation with Sub-Image Adaptability [53.2509590113364]
  Humor, deeply rooted in societal meanings and cultural details, poses a unique challenge for machines.<n>We introduce the textscXMeCap framework, which adopts supervised fine-tuning and reinforcement learning.<n>textscXMeCap achieves an average evaluation score of 75.85 for single-image memes and 66.32 for multi-image memes, outperforming the best baseline by 6.75% and 8.56%, respectively.
  arXiv  Detail & Related papers  (2024-07-24T10:51:46Z)
• Understanding Figurative Meaning through Explainable Visual Entailment [24.831452159672857]
  We propose a new task framing the figurative meaning understanding problem as an explainable visual entailment task.<n>We build the accompanying dataset V-FLUTE, containing 6,027 image, caption, label, explanation instances.<n>We find that VLMs struggle to generalize from literal to figurative meaning, particularly when it is present in images.
  arXiv  Detail & Related papers  (2024-05-02T17:07:25Z)
• Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semantics [50.982315553104975]
  We investigate the bottom-up evolution of lexical semantics for a popular large language model, namely Llama2. Our experiments show that the representations in lower layers encode lexical semantics, while the higher layers, with weaker semantic induction, are responsible for prediction. This is in contrast to models with discriminative objectives, such as mask language modeling, where the higher layers obtain better lexical semantics.
  arXiv  Detail & Related papers  (2024-03-03T13:14:47Z)
• Multi-source Semantic Graph-based Multimodal Sarcasm Explanation Generation [53.97962603641629]
  We propose a novel mulTi-source sEmantic grAph-based Multimodal sarcasm explanation scheme, named TEAM. TEAM extracts the object-level semantic meta-data instead of the traditional global visual features from the input image. TEAM introduces a multi-source semantic graph that comprehensively characterize the multi-source semantic relations.
  arXiv  Detail & Related papers  (2023-06-29T03:26:10Z)
• What do you MEME? Generating Explanations for Visual Semantic Role Labelling in Memes [42.357272117919464]
  We introduce a novel task - EXCLAIM, generating explanations for visual semantic role labeling in memes. To this end, we curate ExHVV, a novel dataset that offers natural language explanations of connotative roles for three types of entities. We also posit LUMEN, a novel multimodal, multi-task learning framework that endeavors to address EXCLAIM optimally.
  arXiv  Detail & Related papers  (2022-12-01T18:21:36Z)
• Do Images really do the Talking? Analysing the significance of Images in Tamil Troll meme classification [0.16863755729554888]
  We try to explore the significance of visual features of images in classifying memes. We try to incorporate the memes as troll and non-trolling memes based on the images and the text on them.
  arXiv  Detail & Related papers  (2021-08-09T09:04:42Z)
• Accurate Word Representations with Universal Visual Guidance [55.71425503859685]
  This paper proposes a visual representation method to explicitly enhance conventional word embedding with multiple-aspect senses from visual guidance. We build a small-scale word-image dictionary from a multimodal seed dataset where each word corresponds to diverse related images. Experiments on 12 natural language understanding and machine translation tasks further verify the effectiveness and the generalization capability of the proposed approach.
  arXiv  Detail & Related papers  (2020-12-30T09:11:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.