Evidence-RL: Towards Evidence-intensive Visual Reasoning

• URL: http://arxiv.org/abs/2608.08021v1
• Date: Sat, 08 Aug 2026 09:02:10 GMT
• Title: Evidence-RL: Towards Evidence-intensive Visual Reasoning
• Abstract summary: Vision-Language Models (VLMs) should answer from concrete image evidence rather than language priors, dataset shortcuts, or irrelevant visual context.<n>We propose Counterfactual Evidence Disentanglement (CED), a training-time evidence audit for VLM grounding.<n>CED neutralizes an object-centric Evidence Region and compares the resulting support drop against matched non-evidence Regions.
• Score: 42.129341199654654
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language Models (VLMs) should answer from concrete image evidence rather than language priors, dataset shortcuts, or irrelevant visual context. Existing perception-aware post-training methods encourage image use through global perturbations or attention proxies, but they do not test whether a sampled answer causally depends on the local evidence that supports it. We propose Counterfactual Evidence Disentanglement (CED), a training-time evidence audit for VLM grounding. For each response, CED neutralizes an object-centric Evidence Region and compares the resulting support drop against matched non-evidence Regions. We combine this signal with answer correctness inside GRPO, rewarding correct answers that rely on the evidence path rather than shortcut or nuisance paths. CED uses weak object-level proposals, requires no question-specific evidence annotations, and adds no inference-time overhead. Across nine public benchmarks and four backbones, CED outperforms prior RL-based post-training methods, with targeted analyses verifying its object-centric signal.

Related papers

• Navigating Sparse Evidence: Agentic Visual RAG via Explicit Context Selection and Consolidation [50.92223196290564]
  We propose a unified agent loop for explicit evidence selection and consolidation.<n>During exploration, SCoRE retains only query-relevant observations and their source pointers in a maintained textual ledger.<n>At termination, it reloads the referenced original images and consolidates the visual evidence for answering, arranging it into a logical sequence.
  arXiv  Detail & Related papers  (2026-09-14T16:11:35Z)
• SAFE-G: Structure-aware Faithful Evidence-guided Generation for Knowledge-based Visual Question Answering [8.538563236020947]
  SAFE-G is a structure-Aware Faithful Evidence-guided Generation framework.<n>It enables precise evidence localization and trustworthy reasoning.<n>It outperforms prior methods by a margin of 8.9% and 3.5%.
  arXiv  Detail & Related papers  (2026-08-22T06:34:53Z)
• Does More Retrieved Evidence Help Visual Retrieval-Augmented Generation with Diffusion Language Models? [22.988311596447435]
  Visual retrieval-augmented generation (RAG) commonly expands the retrieved evidence set to improve answer-page coverage.<n>We show that this assumption does not hold for diffusion language models (DLMs)<n>We propose the Entropy-Based Candidate Filter (ECF), a training-free evidence-admission framework.
  arXiv  Detail & Related papers  (2026-08-07T09:20:40Z)
• Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning [42.17614767170884]
  We show that models extensively copy text from the input into their reasoning traces rather than productively solving the problem.<n>By separating each prompt into task-relevant key evidence and irrelevant distractor context, we show that the root cause is insufficient grounding.<n>Motivated by this diagnosis, we propose GEAR, a reward shaping method that augments the accuracy signal with a grounding reward for overlap with key evidence and a distractor penalty for overlap with irrelevant context.
  arXiv  Detail & Related papers  (2026-07-21T17:59:21Z)
• EG-VQA: Benchmarking Verifiable Video Question Answering with Grounded Temporal Evidence [48.314554375516366]
  This paper introduces the Evidence-Grounded Video Question Answering Benchmark (EG-VQA)<n> EG-VQA is comprised of 2,067 videos and 11,838 QA pairs with fine-grained evidence annotations.<n>We show that even strong proprietary models struggle to accurately ground their predictions.<n>We propose EG-Reasoner, an evidence-grounded reasoning model trained with explicit supervision.
  arXiv  Detail & Related papers  (2026-06-23T16:49:28Z)
• SER: Learning to Ground Video Reasoning with Semantic Evidence Rewards [8.705689021755221]
  We propose Semantic Evidence Reward (SER) to reform RL as a constrained task.<n>Instead of computing pixel-level annotations, SER uses a referee enhancing a VLM to evaluate model-generated evidence claims.<n>On the VLM benchmark, SER achieves 49.6%, improving by 3.0 points over the strong evidence-ground baseline.
  arXiv  Detail & Related papers  (2026-06-23T15:50:33Z)
• Look As You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement Learning [55.232400251303794]
  Look As You Think (LAT) is a reinforcement learning framework that trains models to produce verifiable reasoning paths with consistent attribution.<n>LAT consistently improves the vanilla model in both single- and multi-image settings, yielding average gains of 8.23% in soft exact match (EM) and 47.0% in IoU@0.5.
  arXiv  Detail & Related papers  (2025-11-15T02:50:23Z)
• NeoQA: Evidence-based Question Answering with Generated News Events [53.85274258429368]
  We introduce NeoQA, a benchmark designed to address this issue.<n>We propose our dataset as a new platform for evaluating evidence-based question answering.
  arXiv  Detail & Related papers  (2025-05-09T10:51:29Z)
• From Relevance to Utility: Evidence Retrieval with Feedback for Fact Verification [118.03466985807331]
  We argue that, rather than relevance, for FV we need to focus on the utility that a claim verifier derives from the retrieved evidence.<n>We introduce the feedback-based evidence retriever(FER) that optimize the evidence retrieval process by incorporating feedback from the claim verifier.
  arXiv  Detail & Related papers  (2023-10-18T02:59:38Z)
• Distantly-Supervised Evidence Retrieval Enables Question Answering without Evidence Annotation [19.551623461082617]
  Open-domain question answering answers a question based on evidence retrieved from a large corpus. This paper investigates whether models can learn to find evidence from a large corpus, with only distant supervision from answer labels for model training. We introduce a novel approach (DistDR) that iteratively improves over a weak retriever by alternately finding evidence from the up-to-date model and encouraging the model to learn the most likely evidence.
  arXiv  Detail & Related papers  (2021-10-10T20:01:27Z)
• REM-Net: Recursive Erasure Memory Network for Commonsense Evidence Refinement [130.8875535449478]
  REM-Net is equipped with a module to refine the evidence by erasing the low-quality evidence that does not explain the question answering. Instead of retrieving evidence from existing knowledge bases, REM-Net leverages a pre-trained generative model to generate candidate evidence customized for the question. The results demonstrate the performance of REM-Net and show that the refined evidence is explainable.
  arXiv  Detail & Related papers  (2020-12-24T10:07:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.