MEG-RAG: Quantifying Multi-modal Evidence Grounding for Evidence Selection in RAG

• URL: http://arxiv.org/abs/2604.24564v2
• Date: Thu, 30 Apr 2026 01:34:01 GMT
• Title: MEG-RAG: Quantifying Multi-modal Evidence Grounding for Evidence Selection in RAG
• Abstract summary: Multimodal Retrieval-Augmented Generation (MRAG) addresses key limitations of Multimodal Large Language Models (MLLMs)<n>We propose Multi-modal Evidence Grounding (MEG), a semantic-aware metric that quantifies the contribution of retrieved evidence.<n>We introduce MEG-RAG, a framework that trains a multimodal reranker to align retrieved evidence with the semantic anchors of the ground truth.
• Score: 29.065833225528127
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal Retrieval-Augmented Generation (MRAG) addresses key limitations of Multimodal Large Language Models (MLLMs), such as hallucination and outdated knowledge. However, current MRAG systems struggle to distinguish whether retrieved multimodal data truly supports the semantic core of an answer or merely provides superficial relevance. Existing metrics often rely on heuristic position-based confidence, which fails to capture the informational density of multimodal entities. To address this, we propose Multi-modal Evidence Grounding (MEG), a semantic-aware metric that quantifies the contribution of retrieved evidence. Unlike standard confidence measures, MEG utilizes Semantic Certainty Anchoring, focusing on high-IDF information-bearing tokens that better capture the semantic core of the answer. Building on MEG, we introduce MEG-RAG, a framework that trains a multimodal reranker to align retrieved evidence with the semantic anchors of the ground truth. By prioritizing high-value content based on semantic grounding rather than token probability distributions, MEG-RAG improves the accuracy and multimodal consistency of generated outputs. Extensive experiments on the M$^2$RAG benchmark show that MEG-RAG consistently outperforms strong baselines and demonstrates robust generalization across different teacher models.

Related papers

• Achieving Text-based Person Retrieval with Any Granularity [80.69427598925718]
  This paper introduces a new paradigm, Text-based Person Retrieval with Any Granularity.<n>We formalize a five-level granularity spectrum and construct UFine6926-MG, a high-quality multi-grained annotated dataset.<n>We then propose the Cross-modal Multi-grained Aligning and Matching framework.
  arXiv  Detail & Related papers  (2026-07-23T08:42:25Z)
• MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answering [73.47440561592556]
  We introduce MultAttnAttrib, a training-free attribution-generation method.<n>We show that MultAttnAttrib consistently outperforms a variety of attribution-generation methods.<n>Our method produces attributions at up to one-seventh of the direct inference latency compared to prompting.
  arXiv  Detail & Related papers  (2026-07-01T19:29:19Z)
• MKG-RAG-Bench: Benchmarking Retrieval in Multimodal Knowledge Graph-Augmented Generation [32.81707284618515]
  We introduce MKG-RAG-Bench, a cross-domain benchmark to evaluate retrieval in MKG-RAG.<n>MKG-RAG-Bench is constructed from two multimodal knowledge graphs spanning general and medical domains.<n>We show that effective multimodal retrieval remains challenging yet crucial for end-to-end MKG-RAG performance.
  arXiv  Detail & Related papers  (2026-06-24T23:38:42Z)
• SAMA: Semantic Anchor-aligned Augmentation for Unified Low-Resource Multimodal Information Extraction [12.379572218573365]
  We introduce Semantic Anchor-aligned Multimodal Augmentation (SAMA), a unified framework for generating high-fidelity, task-aware synthetic data.<n>SAMA constructs structured semantic anchors from ground-truth labels to guide a Collaborative Multi-Experts Multimodal Large Language Model.<n>SAMA consistently outperforms state-of-the-art augmentation baselines under both fully supervised and low-resource settings.
  arXiv  Detail & Related papers  (2026-06-17T07:43:33Z)
• OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration [48.11927189422178]
  We investigate multimodal meta-verification, which leverages verifier-generated rationales rather than decision-only signals.<n>We train OmniVerifier-M1, a generalist visual verifier leveraging symbolic meta-verification and decoupled reinforcement learning.<n>This approach paves the way for more reliable, interpretable, and fine-grained multimodal verification, supporting safer and more controllable foundation model deployment.
  arXiv  Detail & Related papers  (2026-05-27T17:56:04Z)
• MASS-RAG: Multi-Agent Synthesis Retrieval-Augmented Generation [36.73029890123542]
  MASS-RAG is a multi-agent synthesis approach to retrieval-augmented generation.<n>It applies distinct agents for evidence summarization, evidence extraction, and reasoning over retrieved documents.<n>It combines their outputs through a dedicated synthesis stage to produce the final answer.
  arXiv  Detail & Related papers  (2026-04-20T17:00:38Z)
• MARA: A Multimodal Adaptive Retrieval-Augmented Framework for Document Question Answering [51.19392014547221]
  Retrieval-based multimodal document QA aims to identify and integrate relevant information from visually rich documents with complex multimodal structures.<n>Current approaches rely on query-agnostic document representations that overlook salient content.<n>We propose the Multimodal Adaptive Retrieval-Augmented (MARA) framework, which introduces query-adaptive generation.
  arXiv  Detail & Related papers  (2026-02-01T12:27:40Z)
• M$^3$KG-RAG: Multi-hop Multimodal Knowledge Graph-enhanced Retrieval-Augmented Generation [20.170643730917963]
  M$3$KG-RAG is a Multi-hop Multimodal Knowledge Graph-enhanced RAG.<n>It retrieves query-aligned audio-visual knowledge from MMKGs.<n>It improves reasoning depth and answer faithfulness in MLLMs.
  arXiv  Detail & Related papers  (2025-12-23T07:54:03Z)
• CoRe-MMRAG: Cross-Source Knowledge Reconciliation for Multimodal RAG [53.950029990391066]
  Cross-source knowledge textbfReconciliation for Multimodal RAG (CoRe-MMRAG)<n>We propose a novel end-to-end framework that effectively reconciles inconsistencies across knowledge sources.<n>Experiments on KB-VQA benchmarks show that CoRe-MMRAG achieves substantial improvements over baseline methods.
  arXiv  Detail & Related papers  (2025-06-03T07:32:40Z)
• VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation [100.06122876025063]
  This paper introduces VisDoMBench, the first comprehensive benchmark designed to evaluate QA systems in multi-document settings.<n>We propose VisDoMRAG, a novel multimodal Retrieval Augmented Generation (RAG) approach that simultaneously utilizes visual and textual RAG.
  arXiv  Detail & Related papers  (2024-12-14T06:24:55Z)
• Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoning [49.3242278912771]
  We introduce a novel multimodal RAG framework named RMR (Retrieval Meets Reasoning) The RMR framework employs a bi-modal retrieval module to identify the most relevant question-answer pairs. It significantly boosts the performance of various vision-language models across a spectrum of benchmark datasets.
  arXiv  Detail & Related papers  (2024-05-31T14:23:49Z)
• Compositional Chain-of-Thought Prompting for Large Multimodal Models [46.721769077885966]
  Compositional Chain-of-Thought (CCoT) is a novel zero-shot Chain-of-Thought prompting method. We first generate an SG using the Large Language Model (LLM) and then use that SG in the prompt to produce a response. We find that the proposed CCoT approach not only improves LMM performance but also improves the performance of several popular LMMs on general multimodal benchmarks.
  arXiv  Detail & Related papers  (2023-11-27T22:23:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.