ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation
- URL: http://arxiv.org/abs/2609.09076v1
- Date: Tue, 08 Sep 2026 17:30:16 GMT
- Title: ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation
- Abstract summary: We introduce ActReview, a rebuttal-guided post-training framework that connects paper-specific diagnoses to concrete, grounded revision plans.<n>Our central insight is that author rebuttals reveal plausible actions for addressing reviewer concerns and can therefore provide latent supervision for revision-oriented feedback.<n>We also introduce ActReview-Bench, a human-curated benchmark of 1,000 instances for evaluating diagnostic quality and revision usefulness.
- Score: 54.82704239643649
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: As LLMs are increasingly used for pre-submission self-review, there is growing demand for feedback that not only identifies weaknesses but also guides authors toward concrete revisions. We study this as Actionable Peer-review Generation and decompose it into two subtasks: diagnostic claim generation and revision suggestion generation. We introduce ActReview, a rebuttal-guided post-training framework that connects paper-specific diagnoses to concrete, grounded revision plans. Our central insight is that author rebuttals reveal plausible actions for addressing reviewer concerns and can therefore provide latent supervision for revision-oriented feedback. From real review-rebuttal threads on OpenReview, we construct ActReview-40K by aligning reviewer weaknesses with author responses and grounding the resulting feedback in localized paper evidence. We post-train Qwen3-8B-Base with multi-task supervised fine-tuning followed by GRPO using candidate-aware, weakness-specific rubric rewards. We also introduce ActReview-Bench, a human-curated benchmark of 1,000 instances for evaluating diagnostic quality and revision usefulness. Experiments show that ActReview outperforms prior specialized review-generation models on actionability and grounding while remaining competitive with strong prompt-based LLMs. Human evaluation confirms improved revision usefulness while revealing a remaining gap in technical accuracy, and additional analyses support generalization to held-out papers and robustness across independent judges.
Related papers
- More Criticism Does Not Make a Better Review: EquiReview-R [5.8331781561663645]
We recast AI-assisted review as evidence-guided refinement of a structured concern set.<n>We introduce EquiReview-R, which resolves existing concerns against localized evidence.<n>On a frozen cohort of previously unseen papers, EquiReview-R satisfies the prespecified non-inferiority criterion for major omission.
arXiv Detail & Related papers (2026-09-03T14:49:24Z) - From Passive Generation to Investigation: A Proactive Scientific Peer Review Agent [52.42081442204093]
We argue that a key limitation is the lack of flexibility to proactively investigate suspicious parts of a paper.<n>We propose ProReviewer, a scientific peer review agent that proactively reviews a paper guided by a maintained, structured review log.<n> Experiments show that ProReviewer with an 8B backbone, trained by supervised fine-tuning and optimized by reinforcement learning, achieves the highest average score across five quality dimensions.
arXiv Detail & Related papers (2026-06-11T13:38:23Z) - CoCoReviewBench: A Completeness- and Correctness-Oriented Benchmark for AI Reviewers [39.97153044457166]
We build category-specific benchmark subsets and skip evaluation when human reviews are missing to strengthen Completeness.<n>We also leverage reviewer--author--meta-review discussions as expert annotations and filter unreliable reviews accordingly to strengthen Correctness.<n>CoCoReviewBench curates 3,900 papers from ICLR and NeurIPS to enable reliable and fine-grained evaluation of AI reviewers.
arXiv Detail & Related papers (2026-05-08T15:44:26Z) - ReviewGrounder: Improving Review Substantiveness with Rubric-Guided, Tool-Integrated Agents [50.27474750319121]
Review text according to paper-specific rubrics derived from official guidelines, the paper's content, and human-written reviews.<n>We introduce REVIEWBENCH, a benchmark evaluating review text according to paper-specific rubrics derived from official guidelines, the paper's content, and human-written reviews.<n>We propose REVIEWGROUNDER, a rubric-guided, tool-integrated multi-agent framework that decomposes reviewing into drafting and grounding stages.
arXiv Detail & Related papers (2026-04-15T16:33:04Z) - FactReview: Evidence-Grounded Reviews with Literature Positioning and Execution-Based Claim Verification [57.196748998757954]
We present FactReview, an evidence-grounded reviewing system that combines claim extraction, literature positioning, and execution-based claim verification.<n>Given a submission, FactReview identifies major claims and reported results, retrieves nearby work to clarify the paper's technical position, and, when code is available, executes the released repository.<n>It then produces a concise review and an evidence report that assigns each major claim one of five labels: Supported, Supported by the paper, Partially supported, In conflict, or Inconclusive.
arXiv Detail & Related papers (2026-04-05T11:45:22Z) - RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation [47.274230235946625]
Many AI-generated reviews are superficial and insufficiently actionable, leaving authors without concrete, implementable guidance and motivating the gap this work addresses.<n>We propose RbtAct, which targets actionable review feedback generation and places existing peer review rebuttal at the center of learning.
arXiv Detail & Related papers (2026-03-10T14:30:55Z) - LazyReview A Dataset for Uncovering Lazy Thinking in NLP Peer Reviews [74.87393214734114]
This work introduces LazyReview, a dataset of peer-review sentences annotated with fine-grained lazy thinking categories.<n>Large Language Models (LLMs) struggle to detect these instances in a zero-shot setting.<n> instruction-based fine-tuning on our dataset significantly boosts performance by 10-20 performance points.
arXiv Detail & Related papers (2025-04-15T10:07:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.