MetaphorVU: Towards Metaphorical Video Understanding

• URL: http://arxiv.org/abs/2605.25461v1
• Date: Mon, 25 May 2026 06:12:19 GMT
• Title: MetaphorVU: Towards Metaphorical Video Understanding
• Abstract summary: We propose MetaphorVU-Bench, the first systematic and comprehensive benchmark dedicated to metaphorical video understanding.<n>We find current MLLMs struggle with accurate metaphorical video understanding, lagging far behind human level, primarily due to defective cross-domain mapping.<n>Motivated by this finding, we construct a metaphor knowledge graph as mapping augmentation and propose MetaphorBoost, an inference-time enhancement framework.
• Score: 81.48044445740618
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Metaphorical videos are prevalent across various real-world scenarios to convey complex ideas, and understanding them typically requires high-order cognitive capabilities. The lack of systematic studies on metaphorical video understanding not only constrains the real-world applicability of MLLMs but also impedes the thorough assessment of their high-order cognitive capabilities. To bridge this gap, we propose MetaphorVU-Bench, the first systematic and comprehensive benchmark dedicated to metaphorical video understanding. Through experiments, we find current MLLMs struggle with accurate metaphorical video understanding, lagging far behind human level, primarily due to defective cross-domain mapping. Motivated by this finding, we construct a metaphor knowledge graph as mapping augmentation and propose MetaphorBoost, an inference-time enhancement framework achieving consistent performance improvement. Our benchmark, analysis, and method provide useful insights and a foundation for future research on advancing MLLMs.

Related papers

• Watch, Remember, Reason: Human-View Video Understanding with MLLMs [115.44608894992399]
  Video understanding is being rapidly transformed by large language models (MLLMs)<n>This work presents a human-view on LLM-based video understanding.<n>Rather than treating tasks as isolated benchmarks, this view provides a unified structure for analyzing how MLLMs acquire evidence.
  arXiv  Detail & Related papers  (2026-06-05T16:29:13Z)
• Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling [68.14113731953971]
  This paper introduces MILO, an Implicit spatIaL wOrld modeling paradigm that simulates human-like imagination.<n>We show that our approach significantly enhances spatial reasoning capabilities across multiple baselines and benchmarks.
  arXiv  Detail & Related papers  (2025-12-01T16:01:41Z)
• Unveiling LLMs' Metaphorical Understanding: Exploring Conceptual Irrelevance, Context Leveraging and Syntactic Influence [40.32545329527664]
  Large Language Models (LLMs) demonstrate advanced capabilities in knowledge integration, contextual reasoning, and creative generation.<n>This study examines LLMs' metaphor-processing abilities from three perspectives.
  arXiv  Detail & Related papers  (2025-10-05T09:45:51Z)
• Metaphor identification using large language models: A comparison of RAG, prompt engineering, and fine-tuning [0.6524460254566904]
  This study investigates the potential of large language models (LLMs) to automate metaphor identification in full texts.<n>We compare three methods: (i) retrieval-augmented generation (RAG), where the model is provided with a codebook and instructed to annotate texts based on its rules and examples; (ii) prompt engineering, where we design task-specific verbal instructions; and (iii) fine-tuning, where the model is trained on hand-coded texts to optimize performance.
  arXiv  Detail & Related papers  (2025-09-29T14:50:18Z)
• MESH -- Understanding Videos Like Human: Measuring Hallucinations in Large Video Models [56.49314029765706]
  We introduce MESH, a benchmark designed to evaluate hallucinations in LVMs systematically.<n>MESH uses a Question-Answering framework with binary and multi-choice formats incorporating target and trap instances.<n>We demonstrate that MESH offers an effective and comprehensive approach for identifying hallucinations in videos.
  arXiv  Detail & Related papers  (2025-09-10T12:34:07Z)
• Towards Multimodal Metaphor Understanding: A Chinese Dataset and Model for Metaphor Mapping Identification [9.08615188602226]
  We develop a Chinese multimodal metaphor advertisement dataset (namely CM3D) that includes annotations of specific target and source domains.<n>We propose a Chain-of-NLP (CoT) Prompting-based Metaphor Mapping Identification Model (CPMMIM) which simulates the human cognitive process for identifying these mappings.
  arXiv  Detail & Related papers  (2025-01-05T04:15:03Z)
• A Dual-Perspective Metaphor Detection Framework Using Large Language Models [29.18537460293431]
  We propose DMD, a novel dual-perspective framework for metaphor detection.<n>It harnesses both implicit and explicit applications of metaphor theories to guide LLMs in metaphor detection.<n>In comparison to previous methods, our framework offers more transparent reasoning processes and delivers more reliable predictions.
  arXiv  Detail & Related papers  (2024-12-23T06:50:04Z)
• Science is Exploration: Computational Frontiers for Conceptual Metaphor Theory [0.0]
  We show that Large Language Models (LLMs) can accurately identify and explain the presence of conceptual metaphors in natural language data. Using a novel prompting technique based on metaphor annotation guidelines, we demonstrate that LLMs are a promising tool for large-scale computational research on conceptual metaphors.
  arXiv  Detail & Related papers  (2024-10-11T17:03:13Z)
• SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge [60.76719375410635]
  We propose a new benchmark (SOK-Bench) consisting of 44K questions and 10K situations with instance-level annotations depicted in the videos. The reasoning process is required to understand and apply situated knowledge and general knowledge for problem-solving. We generate associated question-answer pairs and reasoning processes, finally followed by manual reviews for quality assurance.
  arXiv  Detail & Related papers  (2024-05-15T21:55:31Z)
• Metaphor Generation with Conceptual Mappings [58.61307123799594]
  We aim to generate a metaphoric sentence given a literal expression by replacing relevant verbs. We propose to control the generation process by encoding conceptual mappings between cognitive domains. We show that the unsupervised CM-Lex model is competitive with recent deep learning metaphor generation systems.
  arXiv  Detail & Related papers  (2021-06-02T15:27:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.