Conformal Path Reasoning: Trustworthy Knowledge Graph Question Answering via Path-Level Calibration

• URL: http://arxiv.org/abs/2605.08077v1
• Date: Fri, 08 May 2026 17:57:13 GMT
• Title: Conformal Path Reasoning: Trustworthy Knowledge Graph Question Answering via Path-Level Calibration
• Abstract summary: We propose Conformal Path Reasoning (CPR), a trustworthy KGQA framework with two key innovations.<n>First, we perform query-level conformal calibration over path-level scores, preserving the exchangeability while generating path prediction sets.<n>Second, we introduce the Residual Conformal Value Network (RCVNet), a lightweight module trained via PUCT-guided exploration to learn discriminative path-level nonconformity scores.
• Score: 31.29584002718744
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowledge Graph Question Answering (KGQA) has shown promise for grounded and interpretable reasoning, yet existing approaches often fail to provide reliable coverage guarantees over retrieved answers. While Conformal Prediction (CP) offers a principled framework for producing prediction sets with statistical guarantees, prior methods suffer from critical limitations in both calibration validity and score discriminability, resulting in violated coverage guarantees and excessively large prediction sets. To address these pitfalls, we propose Conformal Path Reasoning (CPR), a trustworthy KGQA framework with two key innovations. First, we perform query-level conformal calibration over path-level scores, preserving the exchangeability while generating path prediction sets. Second, we introduce the Residual Conformal Value Network (RCVNet), a lightweight module trained via PUCT-guided exploration to learn discriminative path-level nonconformity scores. Experiments on benchmarks show that CPR significantly improves the Empirical Coverage Rate by 34% while reducing average prediction set size by 40% compared to conformal baselines. These results validate the efficacy of CPR in satisfying coverage guarantees with substantially more compact answer sets.

Related papers

• ST-BCP: Tightening Coverage Bound for Backward Conformal Prediction via Non-Conformity Score Transformation [18.272247805086284]
  Conformal Prediction (CP) provides a statistical framework for uncertainty quantification that constructs prediction sets with coverage guarantees.<n>BCP inverts this paradigm by enforcing a predefined upper bound on set size and estimating the resulting coverage guarantee.<n>We introduce ST-BCP, a novel method that introduces a data-dependent transformation of nonconformity scores to narrow the coverage gap.
  arXiv  Detail & Related papers  (2026-02-02T07:18:35Z)
• Distribution-informed Efficient Conformal Prediction for Full Ranking [22.380815981596403]
  Quantifying uncertainty is critical for the safe deployment of ranking models in real-world applications.<n>Recent work offers a rigorous solution using conformal prediction in a full ranking scenario, which aims to construct prediction sets for the absolute ranks of test items.<n>We propose Distribution-informed Conformal Ranking (DCR), which produces efficient prediction sets by deriving the exact distribution of non-conformity scores.
  arXiv  Detail & Related papers  (2026-01-30T16:16:44Z)
• Conditional Coverage Diagnostics for Conformal Prediction [47.93989136542648]
  We show that conditional coverage estimation can be a classification problem.<n>We call the resulting family of metrics excess risk of the target coverage (ERT)<n>We release an open-source package for ERT as well as previous conditional coverage metrics.
  arXiv  Detail & Related papers  (2025-12-12T18:47:39Z)
• Distribution-informed Online Conformal Prediction [53.674678995825666]
  We propose Conformal Optimistic Prediction (COP), an online conformal prediction algorithm incorporating underlying data pattern into the update rule.<n>COP produces tighter prediction sets when predictable pattern exists, while retaining valid coverage guarantees even when estimates are inaccurate.<n>We prove that COP can achieve valid coverage and construct shorter prediction intervals than other baselines.
  arXiv  Detail & Related papers  (2025-12-08T17:51:49Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Predicate-Conditional Conformalized Answer Sets for Knowledge Graph Embeddings [27.138935426612306]
  CondKGCP approximates predicate-conditional coverage guarantees while maintaining compact prediction sets.<n>We prove the theoretical guarantees and demonstrate empirical effectiveness of CondKGCP by comprehensive evaluations.
  arXiv  Detail & Related papers  (2025-05-22T16:33:20Z)
• Rectifying Conformity Scores for Better Conditional Coverage [75.73184036344908]
  We present a new method for generating confidence sets within the split conformal prediction framework.<n>Our method performs a trainable transformation of any given conformity score to improve conditional coverage while ensuring exact marginal coverage.
  arXiv  Detail & Related papers  (2025-02-22T19:54:14Z)
• Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
  It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
  arXiv  Detail & Related papers  (2025-01-17T12:01:56Z)
• Distribution-free uncertainty quantification for classification under label shift [105.27463615756733]
  We focus on uncertainty quantification (UQ) for classification problems via two avenues. We first argue that label shift hurts UQ, by showing degradation in coverage and calibration. We examine these techniques theoretically in a distribution-free framework and demonstrate their excellent practical performance.
  arXiv  Detail & Related papers  (2021-03-04T20:51:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.