ReviewGrounder: Improving Review Substantiveness with Rubric-Guided, Tool-Integrated Agents
- URL: http://arxiv.org/abs/2604.14261v1
- Date: Wed, 15 Apr 2026 16:33:04 GMT
- Title: ReviewGrounder: Improving Review Substantiveness with Rubric-Guided, Tool-Integrated Agents
- Abstract summary: Review text according to paper-specific rubrics derived from official guidelines, the paper's content, and human-written reviews.<n>We introduce REVIEWBENCH, a benchmark evaluating review text according to paper-specific rubrics derived from official guidelines, the paper's content, and human-written reviews.<n>We propose REVIEWGROUNDER, a rubric-guided, tool-integrated multi-agent framework that decomposes reviewing into drafting and grounding stages.
- Score: 50.27474750319121
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The rapid rise in AI conference submissions has driven increasing exploration of large language models (LLMs) for peer review support. However, LLM-based reviewers often generate superficial, formulaic comments lacking substantive, evidence-grounded feedback. We attribute this to the underutilization of two key components of human reviewing: explicit rubrics and contextual grounding in existing work. To address this, we introduce REVIEWBENCH, a benchmark evaluating review text according to paper-specific rubrics derived from official guidelines, the paper's content, and human-written reviews. We further propose REVIEWGROUNDER, a rubric-guided, tool-integrated multi-agent framework that decomposes reviewing into drafting and grounding stages, enriching shallow drafts via targeted evidence consolidation. Experiments on REVIEWBENCH show that REVIEWGROUNDER, using a Phi-4-14B-based drafter and a GPT-OSS-120B-based grounding stage, consistently outperforms baselines with substantially stronger/larger backbones (e.g., GPT-4.1 and DeepSeek-R1-670B) in both alignment with human judgments and rubric-based review quality across 8 dimensions. The code is available \href{https://github.com/EigenTom/ReviewGrounder}{here}.
Related papers
- ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation [54.82704239643649]
We introduce ActReview, a rebuttal-guided post-training framework that connects paper-specific diagnoses to concrete, grounded revision plans.<n>Our central insight is that author rebuttals reveal plausible actions for addressing reviewer concerns and can therefore provide latent supervision for revision-oriented feedback.<n>We also introduce ActReview-Bench, a human-curated benchmark of 1,000 instances for evaluating diagnostic quality and revision usefulness.
arXiv Detail & Related papers (2026-09-08T17:30:16Z) - SurveyReview: A Reviewer-Aligned Benchmark for Survey Evaluators [50.129606229339146]
SurveyReview is a reviewer-aligned, multi-dimensional benchmark and dataset for survey evaluation.<n>We structure authentic peer-review reports by converting free-form comments into four-dimensional scores.<n>We release standardized train/test splits and an evaluation protocol to measure alignment between automatic evaluators and human reviewers.
arXiv Detail & Related papers (2026-08-07T16:11:46Z) - From Passive Generation to Investigation: A Proactive Scientific Peer Review Agent [52.42081442204093]
We argue that a key limitation is the lack of flexibility to proactively investigate suspicious parts of a paper.<n>We propose ProReviewer, a scientific peer review agent that proactively reviews a paper guided by a maintained, structured review log.<n> Experiments show that ProReviewer with an 8B backbone, trained by supervised fine-tuning and optimized by reinforcement learning, achieves the highest average score across five quality dimensions.
arXiv Detail & Related papers (2026-06-11T13:38:23Z) - FactReview: Evidence-Grounded Reviews with Literature Positioning and Execution-Based Claim Verification [57.196748998757954]
We present FactReview, an evidence-grounded reviewing system that combines claim extraction, literature positioning, and execution-based claim verification.<n>Given a submission, FactReview identifies major claims and reported results, retrieves nearby work to clarify the paper's technical position, and, when code is available, executes the released repository.<n>It then produces a concise review and an evidence report that assigns each major claim one of five labels: Supported, Supported by the paper, Partially supported, In conflict, or Inconclusive.
arXiv Detail & Related papers (2026-04-05T11:45:22Z) - RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation [47.274230235946625]
Many AI-generated reviews are superficial and insufficiently actionable, leaving authors without concrete, implementable guidance and motivating the gap this work addresses.<n>We propose RbtAct, which targets actionable review feedback generation and places existing peer review rebuttal at the center of learning.
arXiv Detail & Related papers (2026-03-10T14:30:55Z) - Identifying Aspects in Peer Reviews [59.02879434536289]
We develop a data-driven schema for deriving aspects from a corpus of peer reviews.<n>We introduce a dataset of peer reviews augmented with aspects and show how it can be used for community-level review analysis.
arXiv Detail & Related papers (2025-04-09T14:14:42Z) - DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Process [30.710131188931317]
DeepReview is a framework designed to emulate expert reviewers by incorporating structured analysis, literature retrieval, and evidence-based argumentation.<n>In its best mode, DeepReviewer-14B achieves win rates of 88.21% and 80.20% against GPT-o1 and DeepSeek-R1 in evaluations.
arXiv Detail & Related papers (2025-03-11T15:59:43Z) - How Useful are Reviews for Recommendation? A Critical Review and
Potential Improvements [8.471274313213092]
We investigate a growing body of work that seeks to improve recommender systems through the use of review text.
Our initial findings reveal several discrepancies in reported results, partly due to copying results across papers despite changes in experimental settings or data pre-processing.
Further investigation calls for discussion on a much larger problem about the "importance" of user reviews for recommendation.
arXiv Detail & Related papers (2020-05-25T16:30:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.