ProvenAI: Provenance-Native Traces of Evidence in Generated Answers

• URL: http://arxiv.org/abs/2606.26449v1
• Date: Wed, 24 Jun 2026 23:22:38 GMT
• Title: ProvenAI: Provenance-Native Traces of Evidence in Generated Answers
• Abstract summary: This paper introduces ProvenAI, a framework that decomposes transparency in multi-hop question answering into three independently measurable layers.<n>The system achieves 53.53% answer accuracy alongside a mean citation-fidelity score of 71.55%.<n>We formalise the relationship between the implemented surface proxy and a token-level KL-divergence target through a stated faithfulness condition.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval-augmented systems routinely present citations alongside generated answers, yet a citation does not confirm that the corresponding source meaningfully shaped the output. This paper introduces ProvenAI, a framework that decomposes transparency in multi-hop question answering into three independently measurable layers: answer correctness, citation fidelity against benchmark supporting evidence, and per-document influence under leave-one-resource-out intervention. Targeting the HotpotQA distractor benchmark through a seven-stage pipeline covering data normalisation, retrieval indexing, citation-aware answer generation, attribution auditing, ablation-based influence estimation, batch evaluation, and interactive inspection, ProvenAI evaluates 7,405 validation examples drawn from a canonical corpus of 509,300 passages. The system achieves 53.53% answer accuracy alongside a mean citation-fidelity score of 71.55%, and a worked example surfaces what we call the citation-influence gap: a clean citation audit co-occurring with a profile in which one cited source registers only weak influence while seven uncited sources demonstrably shift the output. We formalise the relationship between the implemented surface proxy and a token-level KL-divergence target through a stated faithfulness condition, ground the framework in causal-mediation analysis and database-provenance theory, and discuss how the three measurement layers compose with cryptographic provenance architectures emerging in autonomous scientific discovery. ProvenAI establishes that meaningful transparency in retrieval-grounded QA requires traceable links across retrieved, cited, and behaviourally influential evidence as three distinct, independently measured layers.

Related papers

• HiEviDR-Bench: A Benchmark for Hierarchical Evidence Aggregation in Deep Research [52.87128503345243]
  We introduce HiEviDR-Bench, a benchmark for evaluating Hierarchical Evidence Aggregation in Deep Research.<n>HiEviDR-Bench represents each instance with an explicit evidence graph that captures evidence selection, cross-source linking, and aggregation.<n>We develop a traceability-oriented evaluation framework with five dimensions: report quality, evidence traceability, citation accuracy, claim verification, and answer correctness.
  arXiv  Detail & Related papers  (2026-07-27T23:50:46Z)
• Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs [0.564046562677526]
  We show how a Lean 4-based tool-calling architecture can be used to govern empirical claims.<n>Every verified output structurally descends from an attested tool call and a kernel-checked chain of valid inference.<n>A formal sidecar makes the target proposition, source scope, evidence boundary, proof obligation, and abstention condition auditable.<n>Over time, typed sidecars in datasets, APIs, public records, and AI-generated documents can amortize this formalization burden into reusable infrastructure.
  arXiv  Detail & Related papers  (2026-07-14T11:33:44Z)
• MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answering [73.47440561592556]
  We introduce MultAttnAttrib, a training-free attribution-generation method.<n>We show that MultAttnAttrib consistently outperforms a variety of attribution-generation methods.<n>Our method produces attributions at up to one-seventh of the direct inference latency compared to prompting.
  arXiv  Detail & Related papers  (2026-07-01T19:29:19Z)
• How Do LLMs Cite? A Mechanistic Interpretation of Attribution in Retrieval-Augmented Generation [3.4562767039451674]
  We show how a large language model decides whether to attach an inline citation while answering a factoid question.<n>We show that amplifying or attenuating only those critical heads and eliminating 69% of spurious ones on PopQA without harming answer accuracy.<n>The results reveal a potential disconnect between the model's apparent reasoning and its internal computational pathway.
  arXiv  Detail & Related papers  (2026-06-09T07:06:13Z)
• Explicit Evidence Grounding via Structured Inline Citation Generation [22.27784103991804]
  FullCite is a framework that generates structured inline citations linking each claim to both its source document and supporting evidence.<n>We assess citation quality and faithfulness along three dimensions: document-level correctness, evidence span identification, and claim-citation faithfulness.
  arXiv  Detail & Related papers  (2026-06-05T10:42:10Z)
• Verified Misguidance: Measuring Structural Citation Failures in Search-Augmented LLMs [8.721564756242431]
  CITETRACE is a dataset that traces the full citation chain from user query through retrieved source to generated answer.<n>We design a three-dimension evaluation framework that scores each citation on intent-purpose alignment, source suitability, and answer-source fidelity.<n>Across our pool, 30.6% of citations distort their sources and 27.1% originate from domain-inappropriate sources.
  arXiv  Detail & Related papers  (2026-05-27T14:54:05Z)
• Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents [0.18762753243053634]
  Large language models (LLMs) power deep research agents that synthesize information from hundreds of web sources into cited reports.<n>Current approaches either trust models to self-cite accurately, risking bias, or employ retrieval-augmented generation (RAG) that does not validate source accessibility, relevance or factual consistency.<n>We introduce the first source attribution evaluation framework that uses a reproducible AST rubric to extract and evaluate inline citations at scale.
  arXiv  Detail & Related papers  (2026-05-07T17:46:45Z)
• FactReview: Evidence-Grounded Reviews with Literature Positioning and Execution-Based Claim Verification [57.196748998757954]
  We present FactReview, an evidence-grounded reviewing system that combines claim extraction, literature positioning, and execution-based claim verification.<n>Given a submission, FactReview identifies major claims and reported results, retrieves nearby work to clarify the paper's technical position, and, when code is available, executes the released repository.<n>It then produces a concise review and an evidence report that assigns each major claim one of five labels: Supported, Supported by the paper, Partially supported, In conflict, or Inconclusive.
  arXiv  Detail & Related papers  (2026-04-05T11:45:22Z)
• PAVE: Premise-Aware Validation and Editing for Retrieval-Augmented LLMs [8.082352336629816]
  PAVE is an inference-time validation layer for evidence-grounded question answering.<n>It decomposes retrieved context into question-conditioned atomic facts, drafts an answer, scores how well that draft is supported by the extracted premises, and revises low-support outputs before finalization.
  arXiv  Detail & Related papers  (2026-03-21T06:23:21Z)
• Look As You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement Learning [55.232400251303794]
  Look As You Think (LAT) is a reinforcement learning framework that trains models to produce verifiable reasoning paths with consistent attribution.<n>LAT consistently improves the vanilla model in both single- and multi-image settings, yielding average gains of 8.23% in soft exact match (EM) and 47.0% in IoU@0.5.
  arXiv  Detail & Related papers  (2025-11-15T02:50:23Z)
• VeriCite: Towards Reliable Citations in Retrieval-Augmented Generation via Rigorous Verification [107.75781898355562]
  We introduce a novel framework, called VeriCite, designed to rigorously validate supporting evidence and enhance answer attribution.<n>We conduct experiments across five open-source LLMs and four datasets, demonstrating that VeriCite can significantly improve citation quality while maintaining the correctness of the answers.
  arXiv  Detail & Related papers  (2025-10-13T13:38:54Z)
• GERE: Generative Evidence Retrieval for Fact Verification [57.78768817972026]
  We propose GERE, the first system that retrieves evidences in a generative fashion. The experimental results on the FEVER dataset show that GERE achieves significant improvements over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-04-12T03:49:35Z)
• AmbiFC: Fact-Checking Ambiguous Claims with Evidence [57.7091560922174]
  We present AmbiFC, a fact-checking dataset with 10k claims derived from real-world information needs. We analyze disagreements arising from ambiguity when comparing claims against evidence in AmbiFC. We develop models for predicting veracity handling this ambiguity via soft labels.
  arXiv  Detail & Related papers  (2021-04-01T17:40:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.