Does More Retrieved Evidence Help Visual Retrieval-Augmented Generation with Diffusion Language Models?

• URL: http://arxiv.org/abs/2608.07006v1
• Date: Fri, 07 Aug 2026 09:20:40 GMT
• Title: Does More Retrieved Evidence Help Visual Retrieval-Augmented Generation with Diffusion Language Models?
• Abstract summary: Visual retrieval-augmented generation (RAG) commonly expands the retrieved evidence set to improve answer-page coverage.<n>We show that this assumption does not hold for diffusion language models (DLMs)<n>We propose the Entropy-Based Candidate Filter (ECF), a training-free evidence-admission framework.
• Score: 22.988311596447435
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual retrieval-augmented generation (RAG) commonly expands the retrieved evidence set to improve answer-page coverage, implicitly assuming that all available evidence should be passed to the generator. We show that this assumption does not hold for diffusion language models (DLMs): retrieving more pages increases answer-page recall, whereas unconditionally passing all retrieved pages to the generator often reduces answer accuracy, primarily because of semantic conflict. A latent-source analysis explains this mismatch through source-coherence loss in parallel denoising, where position-wise proposals can combine incompatible visual sources into unsupported answers. We further find that such interference is already visible in the first-step answer-block distribution, making it possible to assess evidence before decoding. To preserve retrieval coverage while limiting harmful visual exposure, we propose the Entropy-Based Candidate Filter (ECF), a training-free evidence-admission framework. To reduce irrelevant content within individual candidates, ECF constructs multi-granularity evidence units; to identify beneficial additional evidence, it uses blank-controlled block confidence and retrieval rank to determine whether and which candidate should enter the final context. Across three multimodal DLMs and five visual QA benchmarks, ECF improves answer accuracy by 2.62 percentage points on average over the strongest fixed top-$k$ input and, with LLaDA2.0-Uni, by 2.37 percentage points on average over the best competing training-free result for each dataset. These results show that broader retrieval benefits visual DLM-RAG through selective evidence admission rather than unconditional evidence expansion. Code is publicly available at https://github.com/wjkuser/ECF.

Related papers

• Navigating Sparse Evidence: Agentic Visual RAG via Explicit Context Selection and Consolidation [50.92223196290564]
  We propose a unified agent loop for explicit evidence selection and consolidation.<n>During exploration, SCoRE retains only query-relevant observations and their source pointers in a maintained textual ledger.<n>At termination, it reloads the referenced original images and consolidates the visual evidence for answering, arranging it into a logical sequence.
  arXiv  Detail & Related papers  (2026-09-14T16:11:35Z)
• Finding the Right Evidence: Factor-Guided Coarse-to-Fine Reasoning for Long Videos [50.28847179290796]
  PACE is a factor-guided framework for long-video evidence acquisition.<n>On MMR-V with the open-source Qwen3-VL backbone, PACE achieves 42.6% accuracy.
  arXiv  Detail & Related papers  (2026-08-26T19:38:17Z)
• SAFE-G: Structure-aware Faithful Evidence-guided Generation for Knowledge-based Visual Question Answering [8.538563236020947]
  SAFE-G is a structure-Aware Faithful Evidence-guided Generation framework.<n>It enables precise evidence localization and trustworthy reasoning.<n>It outperforms prior methods by a margin of 8.9% and 3.5%.
  arXiv  Detail & Related papers  (2026-08-22T06:34:53Z)
• Evidence-RL: Towards Evidence-intensive Visual Reasoning [42.129341199654654]
  Vision-Language Models (VLMs) should answer from concrete image evidence rather than language priors, dataset shortcuts, or irrelevant visual context.<n>We propose Counterfactual Evidence Disentanglement (CED), a training-time evidence audit for VLM grounding.<n>CED neutralizes an object-centric Evidence Region and compares the resulting support drop against matched non-evidence Regions.
  arXiv  Detail & Related papers  (2026-08-08T09:02:10Z)
• CQC-RAG: Robust Retrieval-Augmented Generation via Cross-Query Consistency [9.50454647525075]
  We introduce CQC-RAG, a framework that co-designs query-level diversity injection with cross-Query Consistency evaluation.<n>CQC-RAG rewrites the original question into diverse but meaning-preserving queries.<n>It applies an evidence-grounded protocol to extract answer-evidence pairs and selects answers according to confidence stability.
  arXiv  Detail & Related papers  (2026-06-11T15:01:49Z)
• Align Documents to Questions: Question-Oriented Document Rewriting for Retrieval-Augmented Generation [51.55755193937205]
  We propose QREAM, a style-controlled rewriter that aligns retrieved documents with a question-oriented style while preserving facts.<n>Our framework consists of two stages: (1) QREAM-ICL, which uses stylistic seeds to guide iterative rewriting exploration; and (2) QREAM-FT, a lightweight student model distilled from denoised ICL outputs.
  arXiv  Detail & Related papers  (2026-04-19T08:39:21Z)
• Benchmarking Deflection and Hallucination in Large Vision-Language Models [25.176271096443482]
  Existing benchmarks overlook conflicts between visual and textual evidence.<n>We introduce VLM-DeflectionBench, a benchmark of 2,775 samples spanning diverse multimodal retrieval settings.<n>Our results highlight the need to evaluate not only what models know, but how they behave when they do not.
  arXiv  Detail & Related papers  (2026-04-13T20:22:22Z)
• Rethinking the Reranker: Boundary-Aware Evidence Selection for Robust Retrieval-Augmented Generation [64.09110141948693]
  Retrieval-Augmented Generation (RAG) systems remain brittle under realistic retrieval noise.<n>We propose BAR-RAG, which reframes the reranker as a boundary-aware evidence selector that targets the generator's Goldilocks Zone.<n>Bar-RAG consistently improves end-to-end performance under noisy retrieval.
  arXiv  Detail & Related papers  (2026-02-03T16:08:23Z)
• GRACE: Reinforcement Learning for Grounded Response and Abstention under Contextual Evidence [9.80421132842862]
  Retrieval-Augmented Generation (RAG) integrates external knowledge to enhance Large Language Models (LLMs)<n>RAG is susceptible to two critical flaws: providing correct answers without explicit grounded evidence and producing fabricated responses when the retrieved context is insufficient.<n>We propose GRACE, a reinforcement-learning framework that simultaneously mitigates both types of flaws.
  arXiv  Detail & Related papers  (2026-01-08T02:47:33Z)
• ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answering [54.72902502486611]
  ReAG is a Reasoning-Augmented Multimodal RAG approach that combines coarse- and fine-grained retrieval with a critic model that filters irrelevant passages.<n>ReAG significantly outperforms prior methods, improving answer accuracy and providing interpretable reasoning grounded in retrieved evidence.
  arXiv  Detail & Related papers  (2025-11-27T19:01:02Z)
• Look As You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement Learning [55.232400251303794]
  Look As You Think (LAT) is a reinforcement learning framework that trains models to produce verifiable reasoning paths with consistent attribution.<n>LAT consistently improves the vanilla model in both single- and multi-image settings, yielding average gains of 8.23% in soft exact match (EM) and 47.0% in IoU@0.5.
  arXiv  Detail & Related papers  (2025-11-15T02:50:23Z)
• VeriCite: Towards Reliable Citations in Retrieval-Augmented Generation via Rigorous Verification [107.75781898355562]
  We introduce a novel framework, called VeriCite, designed to rigorously validate supporting evidence and enhance answer attribution.<n>We conduct experiments across five open-source LLMs and four datasets, demonstrating that VeriCite can significantly improve citation quality while maintaining the correctness of the answers.
  arXiv  Detail & Related papers  (2025-10-13T13:38:54Z)
• DiffusionRet: Generative Text-Video Retrieval with Diffusion Model [56.03464169048182]
  Existing text-video retrieval solutions focus on maximizing the conditional likelihood, i.e., p(candidates|query) We creatively tackle this task from a generative viewpoint and model the correlation between the text and the video as their joint probability p(candidates,query) This is accomplished through a diffusion-based text-video retrieval framework (DiffusionRet), which models the retrieval task as a process of gradually generating joint distribution from noise.
  arXiv  Detail & Related papers  (2023-03-17T10:07:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.