Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval Augmented Generation

• URL: http://arxiv.org/abs/2505.21072v2
• Date: Wed, 28 May 2025 13:05:12 GMT
• Title: Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval Augmented Generation
• Authors: Ekaterina Fadeeva, Aleksandr Rubashevskii, Roman Vashurin, Shehzaad Dhuliawala, Artem Shelmanov, Timothy Baldwin, Preslav Nakov, Mrinmaya Sachan, Maxim Panov,
• Abstract summary: We introduce FRANQ (Faithfulness-based Retrieval Augmented UNcertainty Quantification), a novel method for hallucination detection in RAG outputs.<n>We present a new long-form Question Answering (QA) dataset annotated for both factuality and faithfulness.
• Score: 108.13261761812517
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) enhanced with external knowledge retrieval, an approach known as Retrieval-Augmented Generation (RAG), have shown strong performance in open-domain question answering. However, RAG systems remain susceptible to hallucinations: factually incorrect outputs that may arise either from inconsistencies in the model's internal knowledge or incorrect use of the retrieved context. Existing approaches often conflate factuality with faithfulness to the retrieved context, misclassifying factually correct statements as hallucinations if they are not directly supported by the retrieval. In this paper, we introduce FRANQ (Faithfulness-based Retrieval Augmented UNcertainty Quantification), a novel method for hallucination detection in RAG outputs. FRANQ applies different Uncertainty Quantification (UQ) techniques to estimate factuality based on whether a statement is faithful to the retrieved context or not. To evaluate FRANQ and other UQ techniques for RAG, we present a new long-form Question Answering (QA) dataset annotated for both factuality and faithfulness, combining automated labeling with manual validation of challenging examples. Extensive experiments on long- and short-form QA across multiple datasets and LLMs show that FRANQ achieves more accurate detection of factual errors in RAG-generated responses compared to existing methods.

Related papers

• Bounding Hallucinations: Information-Theoretic Guarantees for RAG Systems via Merlin-Arthur Protocols [40.19713302778418]
  We introduce a training framework that treats the entire RAG pipeline as an interactive proof system.<n>We show that M/A-trained LLMs show improved groundedness, completeness, soundness, and reject behavior.<n>Our results demonstrate that autonomous interactive-proof-style supervision provides a principled and practical path toward reliable RAG systems.
  arXiv  Detail & Related papers  (2025-12-12T14:50:38Z)
• Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders [39.5490415037017]
  Retrieval-Augmented Generation (RAG) improves the factuality of large language models (LLMs) by grounding outputs in retrieved evidence.<n>Existing hallucination detection methods for RAG often rely on large-scale detector training.<n>We introduce RAGLens, a lightweight hallucination detector that accurately flags unfaithful RAG outputs.
  arXiv  Detail & Related papers  (2025-12-09T18:33:22Z)
• ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answering [54.72902502486611]
  ReAG is a Reasoning-Augmented Multimodal RAG approach that combines coarse- and fine-grained retrieval with a critic model that filters irrelevant passages.<n>ReAG significantly outperforms prior methods, improving answer accuracy and providing interpretable reasoning grounded in retrieved evidence.
  arXiv  Detail & Related papers  (2025-11-27T19:01:02Z)
• Abductive Inference in Retrieval-Augmented Language Models: Generating and Validating Missing Premises [0.0]
  We propose a framework that integrates abductive inference into retrieval-augmented LLMs.<n> Experimental results on abductive reasoning and multi-hop QA benchmarks show that our approach improves both answer accuracy and reasoning faithfulness.<n>This work highlights abductive inference as a promising direction for enhancing the robustness and explainability of RAG systems.
  arXiv  Detail & Related papers  (2025-11-06T03:37:24Z)
• JointCQ: Improving Factual Hallucination Detection with Joint Claim and Query Generation [40.64428172310572]
  Current large language models (LLMs) often suffer from hallucination issues, i.e., generating content that appears factual but is actually unreliable.<n>In this work, we introduce JointCQ, a joint claim-and-query generation framework designed to construct an effective and efficient claim-query generator.<n>Our framework leverages elaborately designed evaluation criteria to filter synthesized training data, and finetunes a language model for joint claim extraction and query generation.
  arXiv  Detail & Related papers  (2025-10-22T07:15:37Z)
• RADIANT: Retrieval AugmenteD entIty-context AligNmenT -- Introducing RAG-ability and Entity-Context Divergence [5.066415370344766]
  Retrieval-Augmented Generation (RAG) is a technique to enhance factual accuracy by integrating external knowledge into the generation process.<n>This paper introduces Radiant, a framework that merges RAG with alignment designed to optimize the interplay between retrieved evidence and generated content.
  arXiv  Detail & Related papers  (2025-06-28T21:40:35Z)
• Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generation [52.3707788779464]
  We introduce a novel Jensen-Shannon Divergence driven method to Attribute Response to Context (ARC-JSD)<n>ARC-JSD enables efficient and accurate identification of essential context sentences without additional fine-tuning, gradient-calculation or surrogate modelling.<n> Evaluations on a wide range of RAG benchmarks, such as TyDi QA, Hotpot QA, and Musique, using instruction-tuned LLMs in different scales demonstrate superior accuracy and significant computational efficiency improvements.
  arXiv  Detail & Related papers  (2025-05-22T09:04:03Z)
• Uncertainty Quantification in Retrieval Augmented Question Answering [57.05827081638329]
  We propose to quantify the uncertainty of a QA model via estimating the utility of the passages it is provided with.<n>We train a lightweight neural model to predict passage utility for a target QA model and show that while simple information theoretic metrics can predict answer correctness up to a certain extent, our approach efficiently approximates or outperforms more expensive sampling-based methods.
  arXiv  Detail & Related papers  (2025-02-25T11:24:52Z)
• ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation [91.20492150248106]
  We investigate the internal mechanisms behind unfaithful generation and identify a subset of mid-to-deep feed-forward networks (FFNs) that are disproportionately activated in such cases.<n>We propose Parametric Knowledge Muting through FFN Suppression (ParamMute), a framework that improves contextual faithfulness by suppressing the activation of unfaithfulness-associated FFNs.<n> Experimental results show that ParamMute significantly enhances faithfulness across both CoFaithfulQA and the established ConFiQA benchmark, achieving substantial reductions in reliance on parametric memory.
  arXiv  Detail & Related papers  (2025-02-21T15:50:41Z)
• SUGAR: Leveraging Contextual Confidence for Smarter Retrieval [28.552283701883766]
  We introduce Semantic Uncertainty Guided Adaptive Retrieval (SUGAR)<n>We leverage context-based entropy to actively decide whether to retrieve and to further determine between single-step and multi-step retrieval.<n>Our empirical results show that selective retrieval guided by semantic uncertainty estimation improves the performance across diverse question answering tasks, as well as achieves a more efficient inference.
  arXiv  Detail & Related papers  (2025-01-09T01:24:59Z)
• Atomic Fact Decomposition Helps Attributed Question Answering [30.75332718824254]
  Attributed Question Answering (AQA) aims to provide both a trustworthy answer and a reliable attribution report for a question. This paper proposes an Atomic fact decomposition-based Retrieval and Editing framework. It decomposes the generated long-form answers into molecular clauses and atomic facts by the instruction-tuned LLMs.
  arXiv  Detail & Related papers  (2024-10-22T05:25:54Z)
• DeepNote: Note-Centric Deep Retrieval-Augmented Generation [72.70046559930555]
  Retrieval-Augmented Generation (RAG) mitigates factual errors and hallucinations in Large Language Models (LLMs) for question-answering (QA)<n>We develop DeepNote, an adaptive RAG framework that achieves in-depth and robust exploration of knowledge sources through note-centric adaptive retrieval.
  arXiv  Detail & Related papers  (2024-10-11T14:03:29Z)
• Localizing Factual Inconsistencies in Attributable Text Generation [91.981439746404]
  We introduce QASemConsistency, a new formalism for localizing factual inconsistencies in attributable text generation. We first demonstrate the effectiveness of the QASemConsistency methodology for human annotation. We then implement several methods for automatically detecting localized factual inconsistencies.
  arXiv  Detail & Related papers  (2024-10-09T22:53:48Z)
• CONFLARE: CONFormal LArge language model REtrieval [0.0]
  Retrieval-augmented generation (RAG) frameworks enable large language models (LLMs) to retrieve relevant information from a knowledge base and incorporate it into the context for generating responses. RAG does not guarantee valid responses if retrieval fails to identify the necessary information as the context for response generation. We introduce a four-step framework for applying conformal prediction to quantify retrieval uncertainty in RAG frameworks.
  arXiv  Detail & Related papers  (2024-04-04T02:58:21Z)
• RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation [42.82192656794179]
  Large Language Models (LLMs) exhibit remarkable capabilities but are prone to generating inaccurate or hallucinatory responses. This limitation stems from their reliance on vast pretraining datasets, making them susceptible to errors in unseen scenarios. Retrieval-Augmented Generation (RAG) addresses this by incorporating external, relevant documents into the response generation process.
  arXiv  Detail & Related papers  (2024-03-31T08:58:54Z)
• HiQA: A Hierarchical Contextual Augmentation RAG for Multi-Documents QA [13.000411428297813]
  We present HiQA, an advanced multi-document question-answering (MDQA) framework that integrates cascading metadata into content and a multi-route retrieval mechanism. We also release a benchmark called MasQA to evaluate and research in MDQA.
  arXiv  Detail & Related papers  (2024-02-01T02:24:15Z)
• Learning to Filter Context for Retrieval-Augmented Generation [75.18946584853316]
  Generation models are required to generate outputs given partially or entirely irrelevant passages. FILCO identifies useful context based on lexical and information-theoretic approaches. It trains context filtering models that can filter retrieved contexts at test time.
  arXiv  Detail & Related papers  (2023-11-14T18:41:54Z)
• FactCHD: Benchmarking Fact-Conflicting Hallucination Detection [64.4610684475899]
  FactCHD is a benchmark designed for the detection of fact-conflicting hallucinations from LLMs. FactCHD features a diverse dataset that spans various factuality patterns, including vanilla, multi-hop, comparison, and set operation. We introduce Truth-Triangulator that synthesizes reflective considerations by tool-enhanced ChatGPT and LoRA-tuning based on Llama2.
  arXiv  Detail & Related papers  (2023-10-18T16:27:49Z)
• HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models [81.56455625624041]
  We introduce the first open-source benchmark to utilize external large language models (LLMs) for ASR error correction. The proposed benchmark contains a novel dataset, HyPoradise (HP), encompassing more than 334,000 pairs of N-best hypotheses. LLMs with reasonable prompt and its generative capability can even correct those tokens that are missing in N-best list.
  arXiv  Detail & Related papers  (2023-09-27T14:44:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.