Facet-Level Tracing of Evidence Uncertainty and Hallucination in RAG

• URL: http://arxiv.org/abs/2604.09174v1
• Date: Fri, 10 Apr 2026 09:59:43 GMT
• Title: Facet-Level Tracing of Evidence Uncertainty and Hallucination in RAG
• Abstract summary: Retrieval-Augmented Generation aims to reduce hallucinations by grounding answers in retrieved evidence.<n>Existing evaluations focus on answer-level or passage-level accuracy, offering limited insight into how evidence is used during generation.<n>We introduce a facet-level diagnostics framework for QA that decomposes each input question into atomic reasoning facets.
• Score: 8.18791900871137
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Retrieval-Augmented Generation (RAG) aims to reduce hallucination by grounding answers in retrieved evidence, yet hallucinated answers remain common even when relevant documents are available. Existing evaluations focus on answer-level or passage-level accuracy, offering limited insight into how evidence is used during generation. In this work, we introduce a facet-level diagnostics framework for QA that decomposes each input question into atomic reasoning facets. For each facet, we assess evidence sufficiency and grounding using a structured Facet x Chunk matrix that combines retrieval relevance with natural language inference-based faithfulness scores. To diagnose evidence usage, we analyze three controlled inference modes: Strict RAG, which enforces exclusive reliance on retrieved evidence; Soft RAG, which allows integration of retrieved evidence and parametric knowledge; and LLM-only generation without retrieval. Comparing these modes enables thorough analysis of retrieval-generation misalignment, defined as cases where relevant evidence is retrieved but not correctly integrated during generation. Across medical QA and HotpotQA, we evaluate three open-source and closed-source LLMs (GPT, Gemini, and LLaMA), providing interpretable diagnostics that reveal recurring facet-level failure modes, including evidence absence, evidence misalignment, and prior-driven overrides. Our results demonstrate that hallucinations in RAG systems are driven less by retrieval accuracy and more by how retrieved evidence is integrated during generation, with facet-level analysis exposing systematic evidence override and misalignment patterns that remain hidden under answer-level evaluation.

Related papers

• CLEAR: Cross-Source Evidence Adjudication for Large Language Models in Medicine [7.517737504590624]
  CLEAR is an agentic framework for cross-source evidence adjudication in large language models (LLMs) in medicine.<n> CLEAR independently generates candidate answers from three complementary pathways---parametric knowledge, locally curated corpora, and dynamically retrieved evidence.<n>An aggregation verifier jointly evaluates the candidates, supporting evidence, provenance, and source-quality information to identify agreement and conflict across sources.
  arXiv  Detail & Related papers  (2026-09-14T20:00:16Z)
• Evidence-Aligned Entity Verification for Hallucination Detection in Retrieval-Augmented Generation [17.29201876137025]
  Hallucination detection is crucial for large language models (LLMs)<n>Current detection methods depend on internal signals like uncertainty and self-consistency checks.<n>Retrieval-augmented generation (RAG) has emerged as a promising solution by retrieving relevant evidence at inference time.
  arXiv  Detail & Related papers  (2026-09-08T05:22:29Z)
• SAGE: From Direct Answering to Evidence-Grounded Inference for Chinese Ancient Document Understanding [61.02068339726333]
  We propose SAGE, an evidence-grounded multi-agent framework for Chinese ancient document understanding.<n>SAGE coordinates specialized agents for task-aware planning, tool-mediated evidence acquisition, claim-level verification, and bounded replanning.<n> Experiments on the AncientDoc benchmark show that SAGE consistently outperforms matched direct-answering baselines.
  arXiv  Detail & Related papers  (2026-08-25T03:01:51Z)
• Beyond Relevance: Bayesian Evidence Acquisition for Agentic Whole-Slide Image Reasoning [72.86819994769634]
  Whole-slide image (WSI) reasoning requires an agent to sequentially acquire visual evidence before answering a diagnostic question.<n>We propose BEACON, a plug-and-play agentic framework that reformulates WSI reasoning as a Bayesian evidence acquisition problem.
  arXiv  Detail & Related papers  (2026-08-06T08:40:57Z)
• HiEviDR-Bench: A Benchmark for Hierarchical Evidence Aggregation in Deep Research [52.87128503345243]
  We introduce HiEviDR-Bench, a benchmark for evaluating Hierarchical Evidence Aggregation in Deep Research.<n>HiEviDR-Bench represents each instance with an explicit evidence graph that captures evidence selection, cross-source linking, and aggregation.<n>We develop a traceability-oriented evaluation framework with five dimensions: report quality, evidence traceability, citation accuracy, claim verification, and answer correctness.
  arXiv  Detail & Related papers  (2026-07-27T23:50:46Z)
• Before the Action: Benchmarking LLMs on Prospective Hypothesis Discovery [95.44432473367836]
  Large language models (LLMs) excel at answering pre-specified questions, yet their ability to navigate the open-ended, pre-conclusion stage of discovery remains largely unmeasured.<n>We introduce Prospective Hypothesis Discovery (PHD), which asks models to autonomously construct grounded, discnative, and testable hypothesis spaces from inconclusive evidence.<n>We introduce HypoArena, comprising HypoData, a benchmark of 988 cases across six scientific and analytical domains, and HypoEval, an evaluation framework for open-ended hypothesis sets.
  arXiv  Detail & Related papers  (2026-07-17T08:56:43Z)
• Diagnosing Evidence Utilization in Long-Context and Retrieval-Augmented Language Models under Matched Evidence Conditions [0.0]
  A model may answer from parametric priors, fail to use evidence that is present, or cite relevant text without converting it into the final answer.<n>This paper introduces a four-condition diagnostic protocol for evidence-utilization evaluation.
  arXiv  Detail & Related papers  (2026-06-04T22:44:57Z)
• AuthTrace: Diagnosing Evidence Construction in Thematically Dense Single-Author Corpora [6.956097396264084]
  We introduce AuthTrace, a diagnostic benchmark built on thematically dense single-author corpora.<n>AuthTrace provides explicit quoted evidence, exact fan-in annotation, and a unified pack-level protocol measuring evidence recall, evidence precision, and answer correctness.
  arXiv  Detail & Related papers  (2026-05-25T03:10:52Z)
• When Iterative RAG Beats Ideal Evidence: A Diagnostic Study in Scientific Multi-hop Question Answering [0.2796197251957245]
  We study whether synchronized iterative retrieval and reasoning can surpass an idealized static upper bound (Gold Context) RAG.<n>We benchmark eleven state-of-the-art LLMs under three regimes: (i) No Context, measuring reliance on parametric memory; (ii) Gold Context, where all evidence is supplied at once; and (iii) Iterative RAG, a training-free oracle controller that alternates retrieval, hypothesis refinement, and evidence-aware stopping.<n>Across models, Iterative RAG consistently outperforms Gold Context, with gains up to 25.6 percentage points, especially for non-reasoning
  arXiv  Detail & Related papers  (2026-01-27T17:35:05Z)
• MedTrust-RAG: Evidence Verification and Trust Alignment for Biomedical Question Answering [21.855579328680246]
  We propose MedTrust-Guided Iterative RAG, a framework designed to enhance factual consistency and hallucinations in medical QA.<n>First, it enforces citation-aware reasoning by requiring all generated content to be explicitly grounded in retrieved medical documents.<n>Second, it employs an iterative retrieval-verification process, where a verification agent assesses evidence adequacy.
  arXiv  Detail & Related papers  (2025-10-16T07:59:11Z)
• Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domain [8.094811345546118]
  Retrieval augmented generation (RAG) systems provide a method for factually grounding the responses of a Large Language Model (LLM) by providing retrieved evidence, or context, as support.<n>This design introduces a critical vulnerability: LLMs may absorb and reproduce misinformation present in retrieved evidence.<n>This problem is magnified if retrieved evidence contains adversarial material explicitly intended to promulgate misinformation.
  arXiv  Detail & Related papers  (2025-09-04T00:45:58Z)
• End-to-End Agentic RAG System Training for Traceable Diagnostic Reasoning [52.12425911708585]
  Deep-DxSearch is an agentic RAG system trained end-to-end with reinforcement learning (RL)<n>In Deep-DxSearch, we first construct a large-scale medical retrieval corpus comprising patient records and reliable medical knowledge sources.<n> Experiments demonstrate that our end-to-end RL training framework consistently outperforms prompt-engineering and training-free RAG approaches.
  arXiv  Detail & Related papers  (2025-08-21T17:42:47Z)
• Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval Augmented Generation [108.13261761812517]
  We introduce FRANQ (Faithfulness-based Retrieval Augmented UNcertainty Quantification), a novel method for hallucination detection in RAG outputs.<n>We present a new long-form Question Answering (QA) dataset annotated for both factuality and faithfulness.
  arXiv  Detail & Related papers  (2025-05-27T11:56:59Z)
• Enhancing LLM Generation with Knowledge Hypergraph for Evidence-Based Medicine [22.983780823136925]
  Evidence-based medicine (EBM) plays a crucial role in the application of large language models (LLMs) in healthcare.<n>We propose using LLMs to gather scattered evidence from multiple sources and present a knowledge hypergraph-based evidence management model.<n>Our approach outperforms existing RAG techniques in application domains of interest to EBM, such as medical quizzing, hallucination detection, and decision support.
  arXiv  Detail & Related papers  (2025-03-18T09:17:31Z)
• Perplexity Trap: PLM-Based Retrievers Overrate Low Perplexity Documents [64.43980129731587]
  We propose a causal-inspired inference-time debiasing method called Causal Diagnosis and Correction (CDC)<n>CDC first diagnoses the bias effect of the perplexity and then separates the bias effect from the overall relevance score.<n> Experimental results across three domains demonstrate the superior debiasing effectiveness.
  arXiv  Detail & Related papers  (2025-03-11T17:59:00Z)
• A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detection [63.56136319976554]
  Large Language Models (LLMs) generate hallucinations, which can cause significant damage when deployed for mission-critical tasks. We propose a self-check approach based on reverse validation to detect factual errors automatically in a zero-resource fashion. We empirically evaluate our method and existing zero-resource detection methods on two datasets.
  arXiv  Detail & Related papers  (2023-10-10T10:14:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.