DeepSciVerify: Verifying Scientific Claim--Citation Alignment via LLM-Driven Evidence Escalation

• URL: http://arxiv.org/abs/2605.27710v1
• Date: Tue, 26 May 2026 21:33:29 GMT
• Title: DeepSciVerify: Verifying Scientific Claim--Citation Alignment via LLM-Driven Evidence Escalation
• Abstract summary: We present DeepSciVerify, a two-stage pipeline for scientific claim-citation verification.<n>The system first verifies claims using the abstract and defers uncertain cases, retrieving and analyzing full-text passages only when necessary.
• Score: 39.146761527401424
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Misalignment between claims and their cited evidence is a common failure mode in reports generated by large language models, limiting their reliability in scientific and other high-stakes settings. We present DeepSciVerify, a two-stage pipeline for scientific claim-citation verification that combines abstract-level reasoning with selective escalation to passage-level evidence. The system first verifies claims using the abstract and defers uncertain cases, retrieving and analyzing full-text passages only when necessary. This design leverages complementary behaviors across LLMs, as some models are more conservative while others are more decisive under uncertainty. On the SCitance benchmark, DeepSciVerify achieves 86.7 Micro-F1, outperforming strong abstract-only baselines by +4.5 points while resolving 67% of instances without full-text retrieval. These results suggest that selective evidence escalation improves both accuracy and efficiency in claim-citation verification.

Related papers

• NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment [54.4265627247104]
  Large language models (LLMs) are increasingly used in peer review at major AI conferences.<n>Existing benchmarks assess novelty as a single holistic score.<n>We present NovGauge, a human-anchored benchmark for fine-grained novelty assessment diagnosis.
  arXiv  Detail & Related papers  (2026-09-10T08:34:56Z)
• DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments [51.049999642138]
  We introduce DRNOISE, a 100-task benchmark for answer recovery under misleading evidence.<n>Each task has a unique gold answer supported by two corroborating indirect record chains; the paired noisy condition adds one plausible document that states a conflicting answer directly.<n>Across agents with strong clean-task performance, this single intervention causes 66-88 percentage-point accuracy drops.
  arXiv  Detail & Related papers  (2026-07-19T15:20:40Z)
• FactReview: Evidence-Grounded Reviews with Literature Positioning and Execution-Based Claim Verification [57.196748998757954]
  We present FactReview, an evidence-grounded reviewing system that combines claim extraction, literature positioning, and execution-based claim verification.<n>Given a submission, FactReview identifies major claims and reported results, retrieves nearby work to clarify the paper's technical position, and, when code is available, executes the released repository.<n>It then produces a concise review and an evidence report that assigns each major claim one of five labels: Supported, Supported by the paper, Partially supported, In conflict, or Inconclusive.
  arXiv  Detail & Related papers  (2026-04-05T11:45:22Z)
• What Do Claim Verification Datasets Actually Test? A Reasoning Trace Analysis [34.429649156970015]
  We generate structured reasoning traces for 24K claim-verification examples using GPT-4o-mini.<n>We find that direct evidence extraction dominates, while multi-sentence synthesis and numerical reasoning are severely under-represented.
  arXiv  Detail & Related papers  (2026-04-02T05:54:51Z)
• CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era [51.63024682584688]
  Large language models (LLMs) introduce a new risk: fabricated references that appear plausible but correspond to no real publications.<n>We present the first comprehensive benchmark and detection framework for hallucinated citations in scientific writing.<n>Our framework significantly outperforms prior methods in both accuracy and interpretability.
  arXiv  Detail & Related papers  (2026-02-26T19:17:39Z)
• The Alignment Bottleneck in Decomposition-Based Claim Verification [17.197804072440665]
  We introduce a new dataset of real-world complex claims featuring temporally bounded evidence and human-annotated sub-claim evidence spans.<n>We evaluate decomposition under two evidence alignment setups: Sub-claim Aligned Evidence (SAE) and Repeated Claim-level Evidence (SRE)<n>Our results reveal that decomposition brings significant performance improvement only when evidence is granular and strictly aligned.
  arXiv  Detail & Related papers  (2026-02-11T00:02:16Z)
• RIGOURATE: Quantifying Scientific Exaggeration with Evidence-Aligned Claim Evaluation [29.44948404858214]
  RIGOURATE retrieves supporting evidence from a paper's body and assigns each claim an overstatement score.<n>The framework consists of a dataset of over 10K claim-evidence sets from ICLR and NeurIPS papers.
  arXiv  Detail & Related papers  (2026-01-07T19:36:08Z)
• CRAVE: A Conflicting Reasoning Approach for Explainable Claim Verification Using LLMs [15.170312674645535]
  CRAVE is a Conflicting Reasoning Approach for explainable claim VErification.<n>It can verify complex claims based on the conflicting rationales reasoned by large language models.<n>CRAVE achieves much better performance than state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-04-21T07:20:31Z)
• Read it Twice: Towards Faithfully Interpretable Fact Verification by Revisiting Evidence [59.81749318292707]
  We propose a fact verification model named ReRead to retrieve evidence and verify claim. The proposed system is able to achieve significant improvements upon best-reported models under different settings.
  arXiv  Detail & Related papers  (2023-05-02T03:23:14Z)
• AmbiFC: Fact-Checking Ambiguous Claims with Evidence [57.7091560922174]
  We present AmbiFC, a fact-checking dataset with 10k claims derived from real-world information needs. We analyze disagreements arising from ambiguity when comparing claims against evidence in AmbiFC. We develop models for predicting veracity handling this ambiguity via soft labels.
  arXiv  Detail & Related papers  (2021-04-01T17:40:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.