The Warrant Gap: Claim-Conditioned Re-scoring for Fact-Checking

• URL: http://arxiv.org/abs/2606.24627v1
• Date: Tue, 23 Jun 2026 14:23:56 GMT
• Title: The Warrant Gap: Claim-Conditioned Re-scoring for Fact-Checking
• Abstract summary: We introduce SIFT -- claim-conditioned re-scoring of extracted evidence spans against the full claim.<n>We evaluate on FEVER, SciFact, 5PILS, and DP across four open-source backbones.
• Score: 1.918675896078337
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fact-checking systems built on LLMs achieve high verdict accuracy on standard benchmarks, yet routinely output Supports labels whose cited evidence does not license the claim. Structured decomposition is the natural way to inspect those warrants, but rigid extraction protocols strip the full-claim context that facets need. We introduce SIFT -- claim-conditioned re-scoring of extracted evidence spans against the full claim -- paired with WSP (Warranted Supports Proportion), an automatic NLI check that the cited warrant entails the claim. We evaluate on FEVER, SciFact, 5PILS, and DP across four open-source backbones. SIFT recovers accuracy on cells where naive decomposition costs up to 27.6 points, while raising WSP above direct prompting; WSP itself calibrates against human gold evidence at AUC 0.92 and precision 0.98.

Related papers

• Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs [0.564046562677526]
  We show how a Lean 4-based tool-calling architecture can be used to govern empirical claims.<n>Every verified output structurally descends from an attested tool call and a kernel-checked chain of valid inference.<n>A formal sidecar makes the target proposition, source scope, evidence boundary, proof obligation, and abstention condition auditable.<n>Over time, typed sidecars in datasets, APIs, public records, and AI-generated documents can amortize this formalization burden into reusable infrastructure.
  arXiv  Detail & Related papers  (2026-07-14T11:33:44Z)
• Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAG [6.113159481005088]
  We introduce FORCEBENCH, a contrastive stress test for evidence-force calibration.<n>Each item holds a cited passage fixed and pairs an evidence-calibrated claim with a localized force-raised variant.<n>Headline experiments use a fixed, locality-filtered 198-pair evaluation set.
  arXiv  Detail & Related papers  (2026-05-27T06:47:44Z)
• DeepSciVerify: Verifying Scientific Claim--Citation Alignment via LLM-Driven Evidence Escalation [39.146761527401424]
  We present DeepSciVerify, a two-stage pipeline for scientific claim-citation verification.<n>The system first verifies claims using the abstract and defers uncertain cases, retrieving and analyzing full-text passages only when necessary.
  arXiv  Detail & Related papers  (2026-05-26T21:33:29Z)
• Differentiable Conformal Training for LLM Reasoning Factuality [6.534904345823168]
  Large Language Models (LLMs) frequently hallucinate, limiting their reliability in critical applications.<n>Recent work extends Conformal Prediction to filter out risky claims, ensuring that hallucination rates remain below a user-specified level.<n>We introduce Differentiable Coherent Factuality (DCF), a fully differentiable relaxation that enables learning improved scorers while provably recovering the original algorithm's guarantees.
  arXiv  Detail & Related papers  (2026-04-22T01:35:31Z)
• FactReview: Evidence-Grounded Reviews with Literature Positioning and Execution-Based Claim Verification [57.196748998757954]
  We present FactReview, an evidence-grounded reviewing system that combines claim extraction, literature positioning, and execution-based claim verification.<n>Given a submission, FactReview identifies major claims and reported results, retrieves nearby work to clarify the paper's technical position, and, when code is available, executes the released repository.<n>It then produces a concise review and an evidence report that assigns each major claim one of five labels: Supported, Supported by the paper, Partially supported, In conflict, or Inconclusive.
  arXiv  Detail & Related papers  (2026-04-05T11:45:22Z)
• DeepFact: Co-Evolving Benchmarks and Agents for Deep Research Factuality [49.62610727661819]
  Existing fact-checkers are primarily designed for general-domain, factoid-style atomic claims.<n>We propose Audit-then-Score (AtS), where benchmark labels and rationales are explicitly revisable.<n>We instantiate AtS as DeepFact-Bench, a versioned DRR factuality benchmark with auditable rationales.
  arXiv  Detail & Related papers  (2026-03-06T05:05:57Z)
• Distill and Align Decomposition for Enhanced Claim Verification [51.93960785128124]
  Complex claim verification requires decomposing sentences into verifiable subclaims.<n>We propose a reinforcement learning approach that optimises decomposition quality and verifier alignment.<n>Our framework enables smaller language models to achieve state-of-the-art claim verification.
  arXiv  Detail & Related papers  (2026-02-25T12:32:04Z)
• The Alignment Bottleneck in Decomposition-Based Claim Verification [17.197804072440665]
  We introduce a new dataset of real-world complex claims featuring temporally bounded evidence and human-annotated sub-claim evidence spans.<n>We evaluate decomposition under two evidence alignment setups: Sub-claim Aligned Evidence (SAE) and Repeated Claim-level Evidence (SRE)<n>Our results reveal that decomposition brings significant performance improvement only when evidence is granular and strictly aligned.
  arXiv  Detail & Related papers  (2026-02-11T00:02:16Z)
• Retrieve-Refine-Calibrate: A Framework for Complex Claim Fact-Checking [32.6738019397553]
  We propose a Retrieve-Refine-Calibrate (RRC) framework based on large language models (LLMs)<n>Specifically, the framework first identifies the entities mentioned in the claim and retrieves evidence relevant to them.<n>Then, it refines the retrieved evidence based on the claim to reduce irrelevant information.<n>Finally, it calibrates the verification process by re-evaluating low-confidence predictions.
  arXiv  Detail & Related papers  (2026-01-23T08:48:52Z)
• Contrastive Learning to Improve Retrieval for Real-world Fact Checking [84.57583869042791]
  We present Contrastive Fact-Checking Reranker (CFR), an improved retriever for fact-checking complex claims. We leverage the AVeriTeC dataset, which annotates subquestions for claims with human written answers from evidence documents. We find a 6% improvement in veracity classification accuracy on the dataset.
  arXiv  Detail & Related papers  (2024-10-07T00:09:50Z)
• From Relevance to Utility: Evidence Retrieval with Feedback for Fact Verification [118.03466985807331]
  We argue that, rather than relevance, for FV we need to focus on the utility that a claim verifier derives from the retrieved evidence.<n>We introduce the feedback-based evidence retriever(FER) that optimize the evidence retrieval process by incorporating feedback from the claim verifier.
  arXiv  Detail & Related papers  (2023-10-18T02:59:38Z)
• WiCE: Real-World Entailment for Claims in Wikipedia [63.234352061821625]
  We propose WiCE, a new fine-grained textual entailment dataset built on natural claim and evidence pairs extracted from Wikipedia. In addition to standard claim-level entailment, WiCE provides entailment judgments over sub-sentence units of the claim. We show that real claims in our dataset involve challenging verification and retrieval problems that existing models fail to address.
  arXiv  Detail & Related papers  (2023-03-02T17:45:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.