Towards Multimodal Metaphor Understanding: A Chinese Dataset and Model for Metaphor Mapping Identification

• URL: http://arxiv.org/abs/2501.02434v1
• Date: Sun, 05 Jan 2025 04:15:03 GMT
• Title: Towards Multimodal Metaphor Understanding: A Chinese Dataset and Model for Metaphor Mapping Identification
• Authors: Dongyu Zhang, Shengcheng Yin, Jingwei Yu, Zhiyao Wu, Zhen Li, Chengpei Xu, Xiaoxia Wang, Feng Xia,
• Abstract summary: We develop a Chinese multimodal metaphor advertisement dataset (namely CM3D) that includes annotations of specific target and source domains. We propose a Chain-of-NLP (CoT) Prompting-based Metaphor Mapping Identification Model (CPMMIM) which simulates the human cognitive process for identifying these mappings.
• Score: 9.08615188602226
• License:
• Abstract: Metaphors play a crucial role in human communication, yet their comprehension remains a significant challenge for natural language processing (NLP) due to the cognitive complexity involved. According to Conceptual Metaphor Theory (CMT), metaphors map a target domain onto a source domain, and understanding this mapping is essential for grasping the nature of metaphors. While existing NLP research has focused on tasks like metaphor detection and sentiment analysis of metaphorical expressions, there has been limited attention to the intricate process of identifying the mappings between source and target domains. Moreover, non-English multimodal metaphor resources remain largely neglected in the literature, hindering a deeper understanding of the key elements involved in metaphor interpretation. To address this gap, we developed a Chinese multimodal metaphor advertisement dataset (namely CM3D) that includes annotations of specific target and source domains. This dataset aims to foster further research into metaphor comprehension, particularly in non-English languages. Furthermore, we propose a Chain-of-Thought (CoT) Prompting-based Metaphor Mapping Identification Model (CPMMIM), which simulates the human cognitive process for identifying these mappings. Drawing inspiration from CoT reasoning and Bi-Level Optimization (BLO), we treat the task as a hierarchical identification problem, enabling more accurate and interpretable metaphor mapping. Our experimental results demonstrate the effectiveness of CPMMIM, highlighting its potential for advancing metaphor comprehension in NLP. Our dataset and code are both publicly available to encourage further advancements in this field.

Related papers

• Meta4XNLI: A Crosslingual Parallel Corpus for Metaphor Detection and Interpretation [6.0158981171030685]
  We present a novel parallel dataset for the tasks of metaphor detection and interpretation that contains metaphor annotations in both Spanish and English. We investigate language models' metaphor identification and understanding abilities through a series of monolingual and cross-lingual experiments.
  arXiv  Detail & Related papers  (2024-04-10T14:44:48Z)
• ContrastWSD: Enhancing Metaphor Detection with Word Sense Disambiguation Following the Metaphor Identification Procedure [1.03590082373586]
  We present a RoBERTa-based metaphor detection model that integrates the Metaphor Identification Procedure (MIP) and Word Sense Disambiguation (WSD) By utilizing the word senses derived from a WSD model, our model enhances the metaphor detection process and outperforms other methods. We evaluate our approach on various benchmark datasets and compare it with strong baselines, indicating the effectiveness in advancing metaphor detection.
  arXiv  Detail & Related papers  (2023-09-06T15:41:38Z)
• Multi-source Semantic Graph-based Multimodal Sarcasm Explanation Generation [53.97962603641629]
  We propose a novel mulTi-source sEmantic grAph-based Multimodal sarcasm explanation scheme, named TEAM. TEAM extracts the object-level semantic meta-data instead of the traditional global visual features from the input image. TEAM introduces a multi-source semantic graph that comprehensively characterize the multi-source semantic relations.
  arXiv  Detail & Related papers  (2023-06-29T03:26:10Z)
• DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning [89.92601337474954]
  Pragmatic reasoning plays a pivotal role in deciphering implicit meanings that frequently arise in real-life conversations. We introduce a novel challenge, DiPlomat, aiming at benchmarking machines' capabilities on pragmatic reasoning and situated conversational understanding.
  arXiv  Detail & Related papers  (2023-06-15T10:41:23Z)
• Metaphorical Polysemy Detection: Conventional Metaphor meets Word Sense Disambiguation [9.860944032009847]
  Linguists distinguish between novel and conventional metaphor, a distinction which the metaphor detection task in NLP does not take into account. In this paper, we investigate the limitations of treating conventional metaphors in this way. We develop the first MPD model, which learns to identify conventional metaphors in the English WordNet.
  arXiv  Detail & Related papers  (2022-12-16T10:39:22Z)
• Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languages [6.7126373378083715]
  Large pre-trained language models (PLMs) are assumed to encode metaphorical knowledge useful for NLP systems. We present studies in multiple metaphor detection datasets and in four languages. Our experiments suggest that contextual representations in PLMs do encode metaphorical knowledge, and mostly in their middle layers.
  arXiv  Detail & Related papers  (2022-03-26T19:05:24Z)
• Exploiting BERT For Multimodal Target SentimentClassification Through Input Space Translation [75.82110684355979]
  We introduce a two-stream model that translates images in input space using an object-aware transformer. We then leverage the translation to construct an auxiliary sentence that provides multimodal information to a language model. We achieve state-of-the-art performance on two multimodal Twitter datasets.
  arXiv  Detail & Related papers  (2021-08-03T18:02:38Z)
• Metaphor Generation with Conceptual Mappings [58.61307123799594]
  We aim to generate a metaphoric sentence given a literal expression by replacing relevant verbs. We propose to control the generation process by encoding conceptual mappings between cognitive domains. We show that the unsupervised CM-Lex model is competitive with recent deep learning metaphor generation systems.
  arXiv  Detail & Related papers  (2021-06-02T15:27:05Z)
• Linguistic Structure Guided Context Modeling for Referring Image Segmentation [61.701577239317785]
  We propose a "gather-propagate-distribute" scheme to model multimodal context by cross-modal interaction. Our LSCM module builds a Dependency Parsing Tree Word Graph (DPT-WG) which guides all the words to include valid multimodal context of the sentence.
  arXiv  Detail & Related papers  (2020-10-01T16:03:51Z)
• Referring Image Segmentation via Cross-Modal Progressive Comprehension [94.70482302324704]
  Referring image segmentation aims at segmenting the foreground masks of the entities that can well match the description given in the natural language expression. Previous approaches tackle this problem using implicit feature interaction and fusion between visual and linguistic modalities. We propose a Cross-Modal Progressive (CMPC) module and a Text-Guided Feature Exchange (TGFE) module to effectively address the challenging task.
  arXiv  Detail & Related papers  (2020-10-01T16:02:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.