FactReview: Evidence-Grounded Reviews with Literature Positioning and Execution-Based Claim Verification

• URL: http://arxiv.org/abs/2604.04074v2
• Date: Tue, 07 Apr 2026 17:20:55 GMT
• Title: FactReview: Evidence-Grounded Reviews with Literature Positioning and Execution-Based Claim Verification
• Abstract summary: We present FactReview, an evidence-grounded reviewing system that combines claim extraction, literature positioning, and execution-based claim verification.<n>Given a submission, FactReview identifies major claims and reported results, retrieves nearby work to clarify the paper's technical position, and, when code is available, executes the released repository.<n>It then produces a concise review and an evidence report that assigns each major claim one of five labels: Supported, Supported by the paper, Partially supported, In conflict, or Inconclusive.
• Score: 57.196748998757954
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Peer review in machine learning is under growing pressure from rising submission volume and limited reviewer time. Most LLM-based reviewing systems read only the manuscript and generate comments from the paper's own narrative. This makes their outputs sensitive to presentation quality and leaves them weak when the evidence needed for review lies in related work or released code. We present FactReview, an evidence-grounded reviewing system that combines claim extraction, literature positioning, and execution-based claim verification. Given a submission, FactReview identifies major claims and reported results, retrieves nearby work to clarify the paper's technical position, and, when code is available, executes the released repository under bounded budgets to test central empirical claims. It then produces a concise review and an evidence report that assigns each major claim one of five labels: Supported, Supported by the paper, Partially supported, In conflict, or Inconclusive. In a case study on CompGCN, FactReview reproduces results that closely match those reported for link prediction and node classification, yet also shows that the paper's broader performance claim across tasks is not fully sustained: on MUTAG graph classification, the reproduced result is 88.4%, whereas the strongest baseline reported in the paper remains 92.6%. The claim is therefore only partially supported. More broadly, this case suggests that AI is most useful in peer review not as a final decision-maker, but as a tool for gathering evidence and helping reviewers produce more evidence-grounded assessments. The code is public at https://github.com/DEFENSE-SEU/Review-Assistant.

Related papers

• ReGround: Grounding Reviewer Comments in Multimodal Evidence [65.14860452709777]
  Reviewer comments naturally relate to specific parts of the reviewed paper, yet grounding these comments to the underlying evidence is difficult due to long multimodal documents.<n>We introduce ReGround, a large-scale dataset for reviewer comment grounding that links 10,267 reviewer comments to 16,274 evidence in the original anonymous submission of 3,656 papers.<n>Results show that retrieval over the entire paper content performs poorly, evidence-type inference is a major bottleneck, and multimodal evidence provides complementary signals that text alone misses.
  arXiv  Detail & Related papers  (2026-09-10T12:32:58Z)
• ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation [54.82704239643649]
  We introduce ActReview, a rebuttal-guided post-training framework that connects paper-specific diagnoses to concrete, grounded revision plans.<n>Our central insight is that author rebuttals reveal plausible actions for addressing reviewer concerns and can therefore provide latent supervision for revision-oriented feedback.<n>We also introduce ActReview-Bench, a human-curated benchmark of 1,000 instances for evaluating diagnostic quality and revision usefulness.
  arXiv  Detail & Related papers  (2026-09-08T17:30:16Z)
• From Passive Generation to Investigation: A Proactive Scientific Peer Review Agent [52.42081442204093]
  We argue that a key limitation is the lack of flexibility to proactively investigate suspicious parts of a paper.<n>We propose ProReviewer, a scientific peer review agent that proactively reviews a paper guided by a maintained, structured review log.<n> Experiments show that ProReviewer with an 8B backbone, trained by supervised fine-tuning and optimized by reinforcement learning, achieves the highest average score across five quality dimensions.
  arXiv  Detail & Related papers  (2026-06-11T13:38:23Z)
• Review the Code, Not the Story: A Vision and Protocol for Code-First Peer Review [2.531440973296736]
  This vision and protocol paper proposes code-first peer review.<n>Authors submit executable research artifacts and minimal claim manifests.<n>A venue-controlled AI system builds the environment, executes experiments, audits code paths, maps claims to evidence, and generates a standardized Review Package for human reviewers.
  arXiv  Detail & Related papers  (2026-06-05T01:26:07Z)
• Peerispect: Claim Verification in Scientific Peer Reviews [12.340116689292726]
  Peerispect is an interactive system that operationalizes claim-level verification in peer reviews.<n>Results are presented through a visual interface that highlights evidence directly in the paper.
  arXiv  Detail & Related papers  (2026-04-19T23:40:26Z)
• ReviewGrounder: Improving Review Substantiveness with Rubric-Guided, Tool-Integrated Agents [50.27474750319121]
  Review text according to paper-specific rubrics derived from official guidelines, the paper's content, and human-written reviews.<n>We introduce REVIEWBENCH, a benchmark evaluating review text according to paper-specific rubrics derived from official guidelines, the paper's content, and human-written reviews.<n>We propose REVIEWGROUNDER, a rubric-guided, tool-integrated multi-agent framework that decomposes reviewing into drafting and grounding stages.
  arXiv  Detail & Related papers  (2026-04-15T16:33:04Z)
• CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era [51.63024682584688]
  Large language models (LLMs) introduce a new risk: fabricated references that appear plausible but correspond to no real publications.<n>We present the first comprehensive benchmark and detection framework for hallucinated citations in scientific writing.<n>Our framework significantly outperforms prior methods in both accuracy and interpretability.
  arXiv  Detail & Related papers  (2026-02-26T19:17:39Z)
• RIGOURATE: Quantifying Scientific Exaggeration with Evidence-Aligned Claim Evaluation [29.44948404858214]
  RIGOURATE retrieves supporting evidence from a paper's body and assigns each claim an overstatement score.<n>The framework consists of a dataset of over 10K claim-evidence sets from ICLR and NeurIPS papers.
  arXiv  Detail & Related papers  (2026-01-07T19:36:08Z)
• Incremental Extractive Opinion Summarization Using Cover Trees [81.59625423421355]
  In online marketplaces user reviews accumulate over time, and opinion summaries need to be updated periodically. In this work, we study the task of extractive opinion summarization in an incremental setting. We present an efficient algorithm for accurately computing the CentroidRank summaries in an incremental setting.
  arXiv  Detail & Related papers  (2024-01-16T02:00:17Z)
• When Reviewers Lock Horn: Finding Disagreement in Scientific Peer Reviews [24.875901048855077]
  We introduce a novel task of automatically identifying contradictions among reviewers on a given article. To the best of our knowledge, we make the first attempt to identify disagreements among peer reviewers automatically.
  arXiv  Detail & Related papers  (2023-10-28T11:57:51Z)
• GERE: Generative Evidence Retrieval for Fact Verification [57.78768817972026]
  We propose GERE, the first system that retrieves evidences in a generative fashion. The experimental results on the FEVER dataset show that GERE achieves significant improvements over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-04-12T03:49:35Z)
• Assessing Effectiveness of Using Internal Signals for Check-Worthy Claim Identification in Unlabeled Data for Automated Fact-Checking [6.193231258199234]
  This paper explores methodology to identify check-worthy claim sentences from fake news articles. We leverage two internal supervisory signals - headline and the abstractive summary - to rank the sentences. We show that while the headline has more gisting similarity with how a fact-checking website writes a claim, the summary-based pipeline is the most promising for an end-to-end fact-checking system.
  arXiv  Detail & Related papers  (2021-11-02T16:17:20Z)
• AmbiFC: Fact-Checking Ambiguous Claims with Evidence [57.7091560922174]
  We present AmbiFC, a fact-checking dataset with 10k claims derived from real-world information needs. We analyze disagreements arising from ambiguity when comparing claims against evidence in AmbiFC. We develop models for predicting veracity handling this ambiguity via soft labels.
  arXiv  Detail & Related papers  (2021-04-01T17:40:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.