SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation

• URL: http://arxiv.org/abs/2605.03534v1
• Date: Tue, 05 May 2026 09:05:40 GMT
• Title: SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation
• Abstract summary: We present SURE-RAG, a transparent aggregation protocol built on the observation that evidence sufficiency is a set-level property.<n>A shared pair-level claim-evidence verifier produces local relation distributions, which SURE-RAG aggregates into interpretable answer-level signals.<n>We evaluate on HotpotQA-RAG v3, a controlled multi-hop benchmark, under an artifact-aware protocol.
• Score: 6.604874054866016
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Retrieval-augmented generation (RAG) grounds answers in retrieved passages, but retrieval is not verification: a passage can be topical and still fail to justify the answer. We frame this gap as evidence sufficiency verification for selective RAG answering: given a question, a candidate answer, and retrieved evidence, predict whether the evidence supports, refutes, or is insufficient, and abstain unless support is established. We present SURE-RAG, a transparent aggregation protocol built on the observation that evidence sufficiency is a set-level property: missing hops and unresolved conflicts cannot be detected by independent passage scoring. A shared pair-level claim-evidence verifier produces local relation distributions, which SURE-RAG aggregates into interpretable answer-level signals -- coverage, relation strength, disagreement, conflict, and retrieval uncertainty -- yielding a three-way decision and an auditable selective score. We evaluate on HotpotQA-RAG v3, a controlled multi-hop benchmark, under an artifact-aware protocol (shortcut baselines, counterfactual swaps, no-oracle checks, GPT-4o audits). Calibrated SURE-RAG reaches 0.9075 Macro-F1 (0.8951 +/- 0.0069), substantially above DeBERTa mean-pooling (0.6516) and a GPT-4o judge (0.7284), while matching a strong but opaque concat cross-encoder (0.8888 +/- 0.0109) with full auditability. Risk at 30% coverage drops from 0.2588 to 0.1642, a 37% reduction in unsafe answers. To deliberately probe the task boundary, we further contrast SURE-RAG with GPT-4o on HaluBench unsafe detection: the ranking reverses (0.3343 vs 0.7389 unsafe-F1), establishing that controlled sufficiency verification and natural hallucination detection are distinct problems.

Related papers

• The Recall Trap: A Recall-Maximizing Retriever Configuration Reduces Issue Resolution in Fixed-Budget Code Context [45.88028371034407]
  We report a controlled case study in code repair, not a new phenomenon.<n>We toggle one flag (one-chunk-perfile deduplication) on an otherwise identical stack.<n>We map where it holds: it reverses on a lexical BM25 (3.2pp, significant cross-paradigm interaction)<n>We show that at a tight fixed budget, do not hard-deduplicate by file, and A/B packing policies against the task, not the metric flag was tuned.
  arXiv  Detail & Related papers  (2026-08-14T19:22:50Z)
• What Would Fix This RAG Failure? Auditing Counterfactual Response with Paired Evidence Interventions [0.0]
  We introduce Pair-ID, an offline audit that holds one query, retrieval state, and reader constant, then crosses two operations, adding missing support and deleting verified nonsupport, to measure a same-failure counterfactual response vector.<n>A complete funnel over 19,981 benchmark queries identifies 11,105 eligible Qwen failures, from which a prospectively fixed SHA-256 ordering selects 1,200 before generating any sampled response.<n>Results show that evidence sensitivity occurs at meaningful rates in the hash-selected eligible-failure sample, is only partially predictable from the observed failure, and is conditional on the reader.
  arXiv  Detail & Related papers  (2026-08-09T22:43:27Z)
• EvidentialRAG: Quantifying and Mitigating Information Conflict in Multi-Source Retrieval-Augmented Generation via Evidential Deep Learning [0.2730969268472861]
  This paper introduces ERAG, an uncertainty-aware RAG framework that converts retrieved chunks into probabilistic evidence before generation.<n>Experiments on CRAG, ConflictQA, and MuSiQue show that ERAG remains competitive with the strongest matched baseline on standard question answering.
  arXiv  Detail & Related papers  (2026-07-11T21:56:15Z)
• When Confidence Takes the Wrong Path: Diagnosing Retrieval-State Lock-In in RAG [0.0]
  Many black-box uncertainty methods still read agreement among sampled answers as confidence.<n>The problem is recognised in deployed RAG, but it has lacked a name, a measurable signature, and a prevalence bound.<n>We name the failure retrieval-state lock-in and diagnose it by separating the three objects a single confidence score conflates.
  arXiv  Detail & Related papers  (2026-06-22T00:08:00Z)
• X-MADAM-RAG: Diagnosing and Handling Chinese-English Evidence Conflict in Retrieval-Augmented Generation [8.057966659934769]
  Retrieval-augmented generation (RAG) systems may receive evidence that is not merely noisy but mutually contradictory.<n>We study this problem through X-RAMDocs-ZHEN, a controlled Chinese-English benchmark for diagnosing evidence conflict in RAG.<n>X-MADAM-RAG is an interpretable pipeline that decomposes evidence handling into per-document candidate extraction, visible-evidence repair, deterministic candidate grouping, and conflict-aware aggregation.
  arXiv  Detail & Related papers  (2026-06-11T04:59:35Z)
• Auditing CoT Answer-Hijack Patches: Source-Control Certificates with Type-I Guarantees [0.0]
  Chain-of-thought (CoT) answer-hijack templates can flip the final numeric answer of a 7B-8B language model on GSM8K or MATH-500.<n>We show that a successful clean-source patch is often read as evidence that the patched activation carries the recovered content.<n>We construct an audit that turns each candidate patch into a source-control certificate with a pre-registered Type-I guarantee.
  arXiv  Detail & Related papers  (2026-06-03T10:49:43Z)
• Affordance Agent Harness: Verification-Gated Skill Orchestration [45.231685718099264]
  Affordance grounding requires identifying where and how an agent should interact in open-world scenes.<n>We propose Affordance Agent Harness, a closed-loop runtime that unifies skills with an evidence store and cost control.
  arXiv  Detail & Related papers  (2026-05-01T13:45:16Z)
• Guaranteeing Knowledge Integration with Joint Decoding for Retrieval-Augmented Generation [45.13215113490545]
  GuarantRAG is a framework that explicitly decouples reasoning from evidence integration.<n> Experiments on five QA benchmarks demonstrate that GuarantRAG improves accuracy by up to 12.1%.
  arXiv  Detail & Related papers  (2026-04-09T09:52:21Z)
• LatentAudit: Real-Time White-Box Faithfulness Monitoring for Retrieval-Augmented Generation with Verifiable Deployment [19.317475241300397]
  We introduce LatentAudit, a white-box auditor that pools mid-to-late residual-stream activations from an open-weight generator.<n>We show that residual-stream geometry carries a usable faithfulness signal, that this signal survives architecture changes, and that the same rule remains amenable to public verification.
  arXiv  Detail & Related papers  (2026-04-07T02:55:32Z)
• From Black Box to Glass Box: Cross-Model ASR Disagreement to Prioto Review in Ambient AI Scribe Documentation [43.148402136307716]
  Cross-model disagreement among heterogeneous ASR systems can act as a reference-free uncertainty signal.<n>We transcribed 50 publicly available medical audio clips with eight ASR systems spanning commercial APIs and open-source engines.<n>Low-agreement regions were enriched for content disagreements, with the content fraction increasing from 53.9% to 73.9% across quintiles of high-risk mass.
  arXiv  Detail & Related papers  (2026-03-02T13:02:13Z)
• Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG [5.4716896485317195]
  Current stateless defences for multimodal agentic RAG fail to detect adversarial strategies that distribute malicious semantics.<n>We introduce MMA-RAGT, an inference-time control framework governed by a Modular Trust Agent (MTA) that maintains an approximate belief state.
  arXiv  Detail & Related papers  (2026-02-24T23:52:27Z)
• Rethinking the Reranker: Boundary-Aware Evidence Selection for Robust Retrieval-Augmented Generation [64.09110141948693]
  Retrieval-Augmented Generation (RAG) systems remain brittle under realistic retrieval noise.<n>We propose BAR-RAG, which reframes the reranker as a boundary-aware evidence selector that targets the generator's Goldilocks Zone.<n>Bar-RAG consistently improves end-to-end performance under noisy retrieval.
  arXiv  Detail & Related papers  (2026-02-03T16:08:23Z)
• SpatialBench-UC: Uncertainty-Aware Evaluation of Spatial Prompt Following in Text-to-Image Generation [0.0]
  SpatialBench-UC is a small, reproducible benchmark for pairwise spatial relations.<n>We release a benchmark package, versioned prompts, pinned configs, per-sample checker outputs, and report tables.<n>We evaluate three baselines, Stable Diffusion 1.5, SD 1.5 BoxDiff, and SD 1.4 GLIGEN.
  arXiv  Detail & Related papers  (2026-01-19T23:37:10Z)
• Seeing through the Conflict: Transparent Knowledge Conflict Handling in Retrieval-Augmented Generation [12.469991196570106]
  TCR (Transparent Conflict Resolution) disentangles semantic match and factual consistency via dual contrastive encoders.<n>It raises knowledge-gap recovery by +21.4 pp and cuts misleading-context overrides by -29.3 pp, while adding only 0.3% parameters.<n>The signals align with human judgements and expose temporal decision patterns.
  arXiv  Detail & Related papers  (2026-01-11T10:08:49Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs [5.838119242443381]
  We propose a reasoning-trace-augmented RAG framework that adds structured, interpretable reasoning across three stages.<n>A Conflict-Aware Trust-Score (CATS) pipeline is introduced which evaluates groundedness, factual correctness, refusal accuracy, and conflict-behavior alignment.
  arXiv  Detail & Related papers  (2025-12-18T17:27:51Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Retrieval-Augmented Generation with Conflicting Evidence [57.66282463340297]
  Large language model (LLM) agents are increasingly employing retrieval-augmented generation (RAG) to improve the factuality of their responses.<n>In practice, these systems often need to handle ambiguous user queries and potentially conflicting information from multiple sources.<n>We propose RAMDocs (Retrieval with Ambiguity and Misinformation in Documents), a new dataset that simulates complex and realistic scenarios for conflicting evidence for a user query.
  arXiv  Detail & Related papers  (2025-04-17T16:46:11Z)
• Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoning [58.57194301645823]
  Large language models (LLMs) are increasingly integrated into real-world personalized applications.<n>The valuable and often proprietary nature of the knowledge bases used in RAG introduces the risk of unauthorized usage by adversaries.<n>Existing methods that can be generalized as watermarking techniques to protect these knowledge bases typically involve poisoning or backdoor attacks.<n>We propose name for harmless' copyright protection of knowledge bases.
  arXiv  Detail & Related papers  (2025-02-10T09:15:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.