Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation

• URL: http://arxiv.org/abs/2608.30163v1
• Date: Mon, 31 Aug 2026 02:30:11 GMT
• Title: Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation
• Abstract summary: multimodal RAG models exhibit limited accuracy in realistic multi-image scenarios.<n>We introduce Doc-REFRAG, a question-guided framework that compresses visual tokens into coarse chunks and selectively expands question-relevant ones.<n> Experiments on six benchmarks show that Doc-REFRAG outperforms eleven strong baselines.
• Score: 41.46863013116681
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world knowledge resides in multimodal documents, necessitating retrieval-augmented generation (RAG) for accurate question answering. However, existing multimodal RAG models are primarily designed for single-image or closed-document settings and exhibit limited accuracy in realistic multi-image scenarios. Moreover, processing numerous retrieved images incurs substantial computational overhead from irrelevant visual tokens. To address these challenges, we introduce DocLongRAG, a large-scale dataset of 343K question--answer pairs, each associated with an average of 37.4 retrieved images to reflect authentic RAG workflows. Building on this dataset, we propose Doc-REFRAG, a question-guided framework that compresses visual tokens into coarse chunks and selectively expands question-relevant ones via a lightweight RL-based selector. Experiments on six benchmarks show that Doc-REFRAG outperforms eleven strong baselines, achieving state-of-the-art accuracy with significantly lower inference latency. Our resources are available at https://github.com/Collab-Gen/Doc-REFRAG.

Related papers

• DocRetriever: A Plug-and-Play Framework for Multimodal Document Retrieval with Comprehensive Benchmark [48.84943754804533]
  Multimodal documents contain diverse elements, such as tables, figures, and layouts.<n>Current approaches typically combine dense visual embedding models with supervised rerankers to achieve high-precision retrieval.<n>We propose DocRetriever, a plug-and-play framework that enhances visual retrieval via a layout-aware sparse embedding technique.
  arXiv  Detail & Related papers  (2026-05-28T14:50:53Z)
• LFRAG: Layout-oriented Fine-grained Retrieval-Augmented Generation on Multimodal Document Understanding [23.227000200423458]
  We propose layout-oriented Fine-grained Retrieval-Augmented Generation (LFRAG), a novel framework that advances multimodal RAG from page-level to block-level retrieval.<n>With block-level late retrieval, LFRAG enables precise query-content alignment and reduces irrelevant content for downstream generation.<n>Experiments on LFDocQA demonstrate that LFRAG achieves state-of-the-art performance on retrieval tasks, outperforms the best baseline by 7.20% in answer accuracy, and reduces token consumption by 73.07% in generation tasks.
  arXiv  Detail & Related papers  (2026-04-18T05:04:49Z)
• Doc-V*:Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA [71.42483000929614]
  Multi-page Document Visual Question Answering requires reasoning over semantics, layouts, and visual elements in long, visually dense documents.<n>We propose Doc-$V*$, an textbfOCR-free agentic framework that casts multi-page DocVQA as sequential evidence aggregation.
  arXiv  Detail & Related papers  (2026-04-15T11:12:27Z)
• LAD-RAG: Layout-aware Dynamic RAG for Visually-Rich Document Understanding [37.12229829548839]
  We propose LAD-RAG, a novel layout-Aware Dynamic RAG framework.<n>LAD-RAG constructs a symbolic document graph that captures layout structure and cross-page dependencies.<n>Experiments on MMLongBench-Doc, LongDocURL, DUDE, and MP-DocVQA demonstrate that LAD-RAG improves retrieval, achieving over 90% perfect recall on average without any top-k tuning.
  arXiv  Detail & Related papers  (2025-10-08T17:02:04Z)
• Docopilot: Improving Multimodal Models for Document-Level Understanding [87.60020625241178]
  We present a high-quality document-level dataset, Doc-750K, designed to support in-depth understanding of multimodal documents.<n>This dataset includes diverse document structures, extensive cross-page dependencies, and real question-answer pairs derived from the original documents.<n>Building on the dataset, we develop a native multimodal model, Docopilot, which can accurately handle document-level dependencies without relying on RAG.
  arXiv  Detail & Related papers  (2025-07-19T16:03:34Z)
• Hierarchical Lexical Graph for Enhanced Multi-Hop Retrieval [22.33550491040999]
  RAG grounds large language models in external evidence, yet it still falters when answers must be pieced together across semantically distant documents.<n>We build two plug-and-play retrievers: StatementGraphRAG and TopicGraphRAG.<n>Our methods outperform naive chunk-based RAG achieving an average relative improvement of 23.1% in retrieval recall and correctness.
  arXiv  Detail & Related papers  (2025-06-09T17:58:35Z)
• Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answering [60.062194349648195]
  Document Visual Question Answering (DocVQA) faces dual challenges in processing lengthy multimodal documents.<n>Current document retrieval-augmented generation (DocRAG) methods remain limited by their text-centric approaches.<n>We introduce MMDocRAG, a comprehensive benchmark featuring 4,055 expert-annotated QA pairs with multi-page, cross-modal evidence chains.
  arXiv  Detail & Related papers  (2025-05-22T09:52:57Z)
• Towards Text-Image Interleaved Retrieval [49.96332254241075]
  We introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences.<n>We construct a TIIR benchmark based on naturally interleaved wikiHow tutorials, where a specific pipeline is designed to generate interleaved queries.<n>We propose a novel Matryoshka Multimodal Embedder (MME), which compresses the number of visual tokens at different granularity.
  arXiv  Detail & Related papers  (2025-02-18T12:00:47Z)
• Multi-Head RAG: Solving Multi-Aspect Problems with LLMs [18.48202014877111]
  Multi-Head RAG (MRAG) is a novel scheme for fetching multi-aspect documents.<n>We show MRAG's design advantages over 18 RAG baselines, empirical improvements of up to 20% in retrieval success ratios.
  arXiv  Detail & Related papers  (2024-06-07T16:59:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.