When to Trust: A Causality-Aware Calibration Framework for Accurate Knowledge Graph Retrieval-Augmented Generation

• URL: http://arxiv.org/abs/2601.09241v1
• Date: Wed, 14 Jan 2026 07:22:59 GMT
• Title: When to Trust: A Causality-Aware Calibration Framework for Accurate Knowledge Graph Retrieval-Augmented Generation
• Abstract summary: Knowledge Graph Retrieval-Augmented Generation (KG-RAG) extends the RAG paradigm by incorporating structured knowledge from knowledge graphs.<n>Existing KG-RAG models are often severely overconfident, producing high-confidence predictions even when retrieved sub-graphs are incomplete or unreliable.<n>We propose Ca2KG, a Causality-aware framework for KG-RAG, which integrates counterfactual prompting, with a panel-based re-scoring mechanism.
• Score: 9.05691591939794
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Knowledge Graph Retrieval-Augmented Generation (KG-RAG) extends the RAG paradigm by incorporating structured knowledge from knowledge graphs, enabling Large Language Models (LLMs) to perform more precise and explainable reasoning. While KG-RAG improves factual accuracy in complex tasks, existing KG-RAG models are often severely overconfident, producing high-confidence predictions even when retrieved sub-graphs are incomplete or unreliable, which raises concerns for deployment in high-stakes domains. To address this issue, we propose Ca2KG, a Causality-aware Calibration framework for KG-RAG. Ca2KG integrates counterfactual prompting, which exposes retrieval-dependent uncertainties in knowledge quality and reasoning reliability, with a panel-based re-scoring mechanism that stabilises predictions across interventions. Extensive experiments on two complex QA datasets demonstrate that Ca2KG consistently improves calibration while maintaining or even enhancing predictive accuracy.

Related papers

• The Confidence Trap: Calibration Attacks for Graph Neural Networks [30.116997325760945]
  We present a framework for studying calibration attacks on graphs.<n>We show that models with higher accuracy or trained on datasets with more classes are more susceptible under this threat model.<n>Our code is publicly available at https://github.com/CaptainCuong/Graph-Calibration-Attack.git.
  arXiv  Detail & Related papers  (2026-06-07T06:16:58Z)
• Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration [72.0672328514289]
  Multimodal learning often grapples with the challenge of low-quality data.<n>We propose a unified framework, termed Conformal Predictive Self-Calibration.<n>Our framework consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-05T14:48:52Z)
• URAG: A Benchmark for Uncertainty Quantification in Retrieval-Augmented Large Language Models [35.441039437111606]
  URAG is a benchmark designed to assess the uncertainty of RAG systems across various fields like healthcare, programming, science, math, and general text.<n>We apply the evaluation pipeline to 8 standard RAG methods, measuring their performance through both accuracy and prediction-set sizes based on LAC and APS metrics.
  arXiv  Detail & Related papers  (2026-03-02T00:22:06Z)
• QualiRAG: Retrieval-Augmented Generation for Visual Quality Understanding [80.66379018208568]
  Visual quality assessment is shifting from prediction to interpretable quality understanding.<n>Current approaches rely on supervised fine-tuning or reinforcement learning on curated instruction.<n>We propose VbfQualiRAG, a framework that systematically leverages latent perceptual knowledge of large multimodal models for visual quality perception.
  arXiv  Detail & Related papers  (2026-01-26T06:27:03Z)
• Explainable Knowledge Graph Retrieval-Augmented Generation (KG-RAG) with KG-SMILE [0.03908906925636173]
  We develop a method-agnostic, perturbation-based framework that provides token and component-level interoperability for Graph RAG using SMILE.<n> KG-SMILE identifies the graph entities and relations most influential to generated outputs, thereby making RAG more transparent.<n>Our findings show that KG-SMILE produces stable, human-aligned explanations, demonstrating its capacity to balance model effectiveness with interpretability.
  arXiv  Detail & Related papers  (2025-09-03T18:29:30Z)
• KG-Infused RAG: Augmenting Corpus-Based RAG with External Knowledge Graphs [58.12674907593879]
  KG-Infused RAG is a framework that incorporates pre-existing large-scale knowledge graphs into RAG.<n> KG-Infused RAG directly performs spreading activation over external KGs to retrieve relevant structured knowledge.<n>Experiments show that KG-Infused RAG consistently outperforms vanilla RAG.
  arXiv  Detail & Related papers  (2025-06-11T09:20:02Z)
• Diagnosing and Addressing Pitfalls in KG-RAG Datasets: Toward More Reliable Benchmarking [63.84117489519164]
  Knowledge Graph Question Answering systems rely on high-quality benchmarks to evaluate complex multi-hop reasoning.<n>Despite their widespread use, popular datasets such as WebQSP and CWQ suffer from critical quality issues.<n>We introduce KGQAGen, an LLM-in-the-loop framework that systematically resolves these pitfalls.<n>Our findings advocate for more rigorous benchmark construction and position KGQAGen as a scalable framework for advancing KGQA evaluation.
  arXiv  Detail & Related papers  (2025-05-29T14:44:52Z)
• Know3-RAG: A Knowledge-aware RAG Framework with Adaptive Retrieval, Generation, and Filtering [9.814926166669366]
  We propose Know3-RAG, a knowledge-aware RAG framework that leverages structured knowledge from knowledge graphs (KGs) to guide three core stages of the RAG process, including retrieval, generation, and filtering.<n> Experiments on multiple open-domain QA benchmarks demonstrate that Know3-RAG consistently outperforms strong baselines, significantly reducing hallucinations and enhancing answer reliability.
  arXiv  Detail & Related papers  (2025-05-19T03:25:18Z)
• DO-RAG: A Domain-Specific QA Framework Using Knowledge Graph-Enhanced Retrieval-Augmented Generation [4.113142669523488]
  Domain-specific QA systems require generative fluency but high factual accuracy grounded in structured expert knowledge.<n>We propose DO-RAG, a scalable and customizable hybrid QA framework that integrates multi-level knowledge graph construction with semantic vector retrieval.
  arXiv  Detail & Related papers  (2025-05-17T06:40:17Z)
• MES-RAG: Bringing Multi-modal, Entity-Storage, and Secure Enhancements to RAG [65.0423152595537]
  We propose MES-RAG, which enhances entity-specific query handling and provides accurate, secure, and consistent responses.<n>MES-RAG introduces proactive security measures that ensure system integrity by applying protections prior to data access.<n> Experimental results demonstrate that MES-RAG significantly improves both accuracy and recall, highlighting its effectiveness in advancing the security and utility of question-answering.
  arXiv  Detail & Related papers  (2025-03-17T08:09:42Z)
• Towards Trustworthy Knowledge Graph Reasoning: An Uncertainty Aware Perspective [17.39302677990171]
  We propose a new trustworthy KG-LLM framework, Uncertainty Aware Knowledge-Graph Reasoning (UAG) We design an uncertainty-aware multi-step reasoning framework that leverages conformal prediction to provide a theoretical guarantee on the prediction set. Our proposed UAG can achieve any pre-defined coverage rate while reducing the prediction set/interval size by 40% on average.
  arXiv  Detail & Related papers  (2024-10-11T16:57:30Z)
• UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generation [93.38604803625294]
  We present UncertaintyRAG, a novel approach for long-context Retrieval-Augmented Generation (RAG) We use Signal-to-Noise Ratio (SNR)-based span uncertainty to estimate similarity between text chunks. UncertaintyRAG outperforms baselines by 2.03% on LLaMA-2-7B, achieving state-of-the-art results.
  arXiv  Detail & Related papers  (2024-10-03T17:39:38Z)
• Explainable Sparse Knowledge Graph Completion via High-order Graph Reasoning Network [111.67744771462873]
  This paper proposes a novel explainable model for sparse Knowledge Graphs (KGs) It combines high-order reasoning into a graph convolutional network, namely HoGRN. It can not only improve the generalization ability to mitigate the information insufficiency issue but also provide interpretability.
  arXiv  Detail & Related papers  (2022-07-14T10:16:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.