ColGraphRAG: Late-Interaction Evidence Retrieval for Multimodal GraphRAG

• URL: http://arxiv.org/abs/2607.16208v1
• Date: Sat, 09 May 2026 08:30:18 GMT
• Title: ColGraphRAG: Late-Interaction Evidence Retrieval for Multimodal GraphRAG
• Abstract summary: We evaluate replacing the visual candidate-ranking operator over graph-linked image nodes with late-interaction MaxSim-style multi-vector scoring in the ColBERT/ColPali lineage.<n>On MultimodalQA, this change is associated with improved retrieval-stage point estimates for graph-linked image candidates and downstream QA gains.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph-grounded multimodal question answering organizes text, tables, and images in a structured evidence graph, yet end-to-end accuracy depends on which multimodal assets are ranked highly enough to enter downstream reasoning; for graph-linked images, single-vector bi-encoder similarity can discard patch- and token-level structure needed for fine-grained alignment. We evaluate replacing the visual candidate-ranking operator over graph-linked image nodes with late-interaction MaxSim-style multi-vector scoring in the ColBERT/ColPali lineage, while keeping offline graph construction, text- and table-side retrieval, structured extraction, and downstream reasoning unchanged. On MultimodalQA, this change is associated with improved retrieval-stage point estimates for graph-linked image candidates and downstream QA gains, with larger movement where visual evidence matters most and mixed trends on text-dominant questions; we interpret the pattern as mechanism-level evidence for graph-linked visual evidence inclusion, while broader validation and finer graph-level diagnostics remain important future work.

Related papers

• GraphLoom: Reliability-Calibrated Graph Evidence Routing for Multimodal KG-RAG [4.035350563640088]
  We present GraphLoom, a reliability-calibrated multimodal knowledge-graph RAG framework for compact and faithful evidence routing.<n>Instead of injecting all retrieved evidence into the generator, GraphLoom performs reliability-aware subgraph retrieval.<n>We show consistent gains in answer quality and evidence faithfulness over strong multimodal RAG, graph-retrieval, and open-source vision-language baselines.
  arXiv  Detail & Related papers  (2026-08-15T06:02:23Z)
• GraphVerse: A Comprehensive Visual Graph Reasoning Benchmark for Multimodal Large Language Models [26.945369697013458]
  GraphVerse is a benchmark that jointly evaluates perception, visual reasoning, and text-based graph reasoning in MLLMs.<n>At its core is a suite of Graph-centric Image Editing strategies that modify graph images while preserving their semantics.<n>We propose VGR-Score, a process-sensitive metric that evaluates reasoning quality beyond final-answer accuracy.
  arXiv  Detail & Related papers  (2026-08-07T03:43:25Z)
• HVM-GraphRAG: Holistic-View Multimodal Graph Retrieval-Augmented Generation on Complex Document [36.46869470798395]
  Multimodal GraphRAG provides a promising direction by organizing document evidence with graph structures.<n>HVM-GraphRAG uses a holistic view to guide graph construction, thereby reducing noisy and conflicting graph updates.<n> Experiments on three datasets show that HVM-GraphRAG achieves the best answer performance in most evaluated settings.
  arXiv  Detail & Related papers  (2026-07-26T12:47:25Z)
• GOMA: Toward Structure-Driven Multimodal Alignment from a Graph Signal Smoothing Perspective [39.44851011793204]
  We propose a structure-driven post-alignment framework that views frozen multimodal embeddings as graph signals.<n>GOMA decouples three key design choices: where messages should flow, how multimodal evidence should propagate, and which smoothing depth should be retained.<n>All experiments follow a transductive MAG retrieval protocol where the graph serves only as unlabeled context and diagonal self-pair edges are removed.
  arXiv  Detail & Related papers  (2026-05-15T08:21:22Z)
• Divide by Question, Conquer by Agent: SPLIT-RAG with Question-Driven Graph Partitioning [62.640169289390535]
  SPLIT-RAG is a multi-agent RAG framework that addresses the limitations with question-driven semantic graph partitioning and collaborative subgraph retrieval.<n>The innovative framework first create Semantic Partitioning of Linked Information, then use the Type-Specialized knowledge base to achieve Multi-Agent RAG.<n>The attribute-aware graph segmentation manages to divide knowledge graphs into semantically coherent subgraphs, ensuring subgraphs align with different query types.<n>A hierarchical merging module resolves inconsistencies across subgraph-derived answers through logical verifications.
  arXiv  Detail & Related papers  (2025-05-20T06:44:34Z)
• RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuning [63.599057862999]
  RefChartQA is a novel benchmark that integrates Chart Question Answering (ChartQA) with visual grounding.<n>Our experiments demonstrate that incorporating spatial awareness via grounding improves response accuracy by over 15%.
  arXiv  Detail & Related papers  (2025-03-29T15:50:08Z)
• Graph-Based Multimodal Contrastive Learning for Chart Question Answering [11.828192162922436]
  This work introduces a novel joint multimodal scene graph framework that explicitly models the relationships among chart components and their underlying structures.<n>The framework integrates both visual and textual graphs to capture structural and semantic characteristics.<n>A graph contrastive learning strategy aligns node representations across modalities enabling their seamless incorporation into a transformer decoder as soft prompts.
  arXiv  Detail & Related papers  (2025-01-08T06:27:07Z)
• Multimodal Multihop Source Retrieval for Web Question Answering [0.0]
  This work deals with the challenge of learning and reasoning over multi-modal multi-hop question answering (QA)<n>We propose a graph reasoning network based on the semantic structure of the sentences to learn multi-source reasoning paths.
  arXiv  Detail & Related papers  (2025-01-07T22:53:56Z)
• InstructG2I: Synthesizing Images from Multimodal Attributed Graphs [50.852150521561676]
  We propose a graph context-conditioned diffusion model called InstructG2I. InstructG2I first exploits the graph structure and multimodal information to conduct informative neighbor sampling. A Graph-QFormer encoder adaptively encodes the graph nodes into an auxiliary set of graph prompts to guide the denoising process.
  arXiv  Detail & Related papers  (2024-10-09T17:56:15Z)
• Deep Prompt Tuning for Graph Transformers [55.2480439325792]
  Fine-tuning is resource-intensive and requires storing multiple copies of large models. We propose a novel approach called deep graph prompt tuning as an alternative to fine-tuning. By freezing the pre-trained parameters and only updating the added tokens, our approach reduces the number of free parameters and eliminates the need for multiple model copies.
  arXiv  Detail & Related papers  (2023-09-18T20:12:17Z)
• Variational Graph Generator for Multi-View Graph Clustering [51.89092260088973]
  We propose Variational Graph Generator for Multi-View Graph Clustering (VGMGC)<n>This generator infers a reliable variational consensus graph based on a priori assumption over multiple graphs.<n>It embeds the inferred view-common graph and view-specific graphs together with features.
  arXiv  Detail & Related papers  (2022-10-13T13:19:51Z)
• Scene Graph Modification as Incremental Structure Expanding [61.84291817776118]
  We focus on scene graph modification (SGM), where the system is required to learn how to update an existing scene graph based on a natural language query. We frame SGM as a graph expansion task by introducing the incremental structure expanding (ISE) We construct a challenging dataset that contains more complicated queries and larger scene graphs than existing datasets.
  arXiv  Detail & Related papers  (2022-09-15T16:26:14Z)
• Interactive Visual Pattern Search on Graph Data via Graph Representation Learning [20.795511688640296]
  We propose a visual analytics system GraphQ to support human-in-the-loop, example-based, subgraph pattern search. To support fast, interactive queries, we use graph neural networks (GNNs) to encode a graph as fixed-length latent vector representation. We also propose a novel GNN for node-alignment called NeuroAlign to facilitate easy validation and interpretation of the query results.
  arXiv  Detail & Related papers  (2022-02-18T22:30:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.