DeepReviewer 2.0: A Traceable Agentic System for Auditable Scientific Peer Review

• URL: http://arxiv.org/abs/2604.09590v1
• Date: Tue, 03 Mar 2026 09:02:17 GMT
• Title: DeepReviewer 2.0: A Traceable Agentic System for Auditable Scientific Peer Review
• Abstract summary: DeepReviewer2.0 is a process-controlled agentic review system built around an output contract.<n>On 134 ICLR2025 submissions under three fixed protocols, an emphun-finetuned 196B model running DeepReviewer2.0 outperforms Gemini-3.1-Pro-preview.
• Score: 22.737940168374546
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Automated peer review is often framed as generating fluent critique, yet reviewers and area chairs need judgments they can \emph{audit}: where a concern applies, what evidence supports it, and what concrete follow-up is required. DeepReviewer~2.0 is a process-controlled agentic review system built around an output contract: it produces a \textbf{traceable review package} with anchored annotations, localized evidence, and executable follow-up actions, and it exports only after meeting minimum traceability and coverage budgets. Concretely, it first builds a manuscript-only claim--evidence--risk ledger and verification agenda, then performs agenda-driven retrieval and writes anchored critiques under an export gate. On 134 ICLR~2025 submissions under three fixed protocols, an \emph{un-finetuned 196B} model running DeepReviewer~2.0 outperforms Gemini-3.1-Pro-preview, improving strict major-issue coverage (37.26\% vs.\ 23.57\%) and winning 71.63\% of micro-averaged blind comparisons against a human review committee, while ranking first among automatic systems in our pool. We position DeepReviewer~2.0 as an assistive tool rather than a decision proxy, and note remaining gaps such as ethics-sensitive checks.

Related papers

• PaperDoctor: Evidence-Grounded and Actionable Feedback for Scientific Papers in Progress [130.58520199343707]
  PaperDoctor is an agent framework for pre-submission feedback with three key innovations.<n>First, a holistic hierarchical framework evaluates writing, layout, references, code, theory, prior work, and experiments.<n>Second, each finding contains an observation, a pointer to specific evidence such as a sentence, equation, or code line, and a revision suggestion.<n>Third, PaperDoctor selectively rebuilds and reruns experiments based on claim importance and compute budget.
  arXiv  Detail & Related papers  (2026-09-15T11:08:11Z)
• ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation [54.82704239643649]
  We introduce ActReview, a rebuttal-guided post-training framework that connects paper-specific diagnoses to concrete, grounded revision plans.<n>Our central insight is that author rebuttals reveal plausible actions for addressing reviewer concerns and can therefore provide latent supervision for revision-oriented feedback.<n>We also introduce ActReview-Bench, a human-curated benchmark of 1,000 instances for evaluating diagnostic quality and revision usefulness.
  arXiv  Detail & Related papers  (2026-09-08T17:30:16Z)
• More Criticism Does Not Make a Better Review: EquiReview-R [5.8331781561663645]
  We recast AI-assisted review as evidence-guided refinement of a structured concern set.<n>We introduce EquiReview-R, which resolves existing concerns against localized evidence.<n>On a frozen cohort of previously unseen papers, EquiReview-R satisfies the prespecified non-inferiority criterion for major omission.
  arXiv  Detail & Related papers  (2026-09-03T14:49:24Z)
• SurveyReview: A Reviewer-Aligned Benchmark for Survey Evaluators [50.129606229339146]
  SurveyReview is a reviewer-aligned, multi-dimensional benchmark and dataset for survey evaluation.<n>We structure authentic peer-review reports by converting free-form comments into four-dimensional scores.<n>We release standardized train/test splits and an evaluation protocol to measure alignment between automatic evaluators and human reviewers.
  arXiv  Detail & Related papers  (2026-08-07T16:11:46Z)
• Precise but Uncoupled: Reviewer Precision Does Not Guarantee Critique Uptake in Multi-Agent Math Reasoning [9.345403722854543]
  We test the assumption that a dedicated review stage should help turn wrong candidates into correct ones.<n>We ask whether this gap is explained by reviewer quality or by whether critique changes the next answer the protocol carries forward.
  arXiv  Detail & Related papers  (2026-07-16T18:38:10Z)
• Articulate Intuition or Genuine Analysis? Benchmarking Epistemic Reliability in LLM-as-a-Judge Peer Reviews [37.684628465527545]
  We operationalise Kahneman's dual-process theory into a structured rubric for peer review and release Kahneman4Review.<n>A benchmark of 3,563 rated reviews scored along nine theoretically motivated textual dimensions, eight bias diagnostics, and a continuous reasoning-quality score.
  arXiv  Detail & Related papers  (2026-07-12T00:03:42Z)
• PaperJury: Due-Process Review for Bounded LaTeX Revision [10.41008524026468]
  PaperJury is a closed-loop review-verdict-revise-verify system built on a deterministic-semantic split.<n>PaperJury combines bounded holistic review, contestability-based routing, a due-process trial, and risk-proportional guard chains for anchor-bounded edits.
  arXiv  Detail & Related papers  (2026-06-15T07:29:34Z)
• From Passive Generation to Investigation: A Proactive Scientific Peer Review Agent [52.42081442204093]
  We argue that a key limitation is the lack of flexibility to proactively investigate suspicious parts of a paper.<n>We propose ProReviewer, a scientific peer review agent that proactively reviews a paper guided by a maintained, structured review log.<n> Experiments show that ProReviewer with an 8B backbone, trained by supervised fine-tuning and optimized by reinforcement learning, achieves the highest average score across five quality dimensions.
  arXiv  Detail & Related papers  (2026-06-11T13:38:23Z)
• Hierarchical Online Prompt Mutation with Dual-Loop Feedback for Guardrailed Evidence Document Generation: A Production-Evaluation Case Study [0.0]
  HOPM is a hierarchical online prompt mutation framework evaluated on a real marketplace dispute-evidence workflow.<n>The paper includes control setup, sample sizes, confidence intervals, paired tests, prompt-token categories, pseudocode, schema, rubric, guardrail taxonomy, and a constructed example.
  arXiv  Detail & Related papers  (2026-05-31T22:17:44Z)
• Beyond Rating: A Comprehensive Evaluation and Benchmark for AI Reviews [69.66583722746904]
  We introduce Beyond Rating, a holistic evaluation framework that assesses AI reviewers across five dimensions.<n>We propose a Max-Recall strategy to accommodate valid expert disagreement and introduce a curated dataset of paper with high-confidence reviews.<n>Our proposed text-centric metrics--particularly the recall of weakness arguments--correlate strongly with rating accuracy.
  arXiv  Detail & Related papers  (2026-04-21T14:21:15Z)
• ReviewGrounder: Improving Review Substantiveness with Rubric-Guided, Tool-Integrated Agents [50.27474750319121]
  Review text according to paper-specific rubrics derived from official guidelines, the paper's content, and human-written reviews.<n>We introduce REVIEWBENCH, a benchmark evaluating review text according to paper-specific rubrics derived from official guidelines, the paper's content, and human-written reviews.<n>We propose REVIEWGROUNDER, a rubric-guided, tool-integrated multi-agent framework that decomposes reviewing into drafting and grounding stages.
  arXiv  Detail & Related papers  (2026-04-15T16:33:04Z)
• FactReview: Evidence-Grounded Reviews with Literature Positioning and Execution-Based Claim Verification [57.196748998757954]
  We present FactReview, an evidence-grounded reviewing system that combines claim extraction, literature positioning, and execution-based claim verification.<n>Given a submission, FactReview identifies major claims and reported results, retrieves nearby work to clarify the paper's technical position, and, when code is available, executes the released repository.<n>It then produces a concise review and an evidence report that assigns each major claim one of five labels: Supported, Supported by the paper, Partially supported, In conflict, or Inconclusive.
  arXiv  Detail & Related papers  (2026-04-05T11:45:22Z)
• BadScientist: Can a Research Agent Write Convincing but Unsound Papers that Fool LLM Reviewers? [21.78901120638025]
  We investigate whether fabrication-oriented paper generation agents can deceive multi-model LLM review systems.<n>Our generator employs presentation-manipulation strategies requiring no real experiments.<n>Despite provably sound aggregation mathematics, integrity checking systematically fails.
  arXiv  Detail & Related papers  (2025-10-20T18:37:11Z)
• FOR-Prompting: From Objection to Revision via an Asymmetric Prompting Protocol [7.765950922513099]
  Reasoning protocols organize internal deliberation but lack an explicit mechanism for external questioning that elicits self-revision.<n>We present FOR-Prompting, an asymmetric protocol where a Defender proposes an answer, an Objectioner raises question-style objections with no direct fixes, and a Host enforces consistency and closure.<n>On GSM8K we observe about a 22% point gain over single-prompt and accuracy on par with CoT, with more than 10% higher ratings in reasoning and coherence from a uniform GPT 4.1 judge.
  arXiv  Detail & Related papers  (2025-10-02T04:57:58Z)
• TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them [58.04324690859212]
  Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks.<n>We identify two fundamental types of inconsistencies: Score-Comparison Inconsistency and Pairwise Transitivity Inconsistency.<n>We propose TrustJudge, a probabilistic framework that addresses these limitations through two key innovations.
  arXiv  Detail & Related papers  (2025-09-25T13:04:29Z)
• Re$^2$: A Consistency-ensured Dataset for Full-stage Peer Review and Multi-turn Rebuttal Discussions [2.5226834810382113]
  We introduce the largest consistency-ensured peer review and rebuttal dataset named Re2.<n>This dataset comprises 19,926 initial submissions, 70,668 review comments, and 53,818 rebuttals from 24 conferences and 21 workshops on OpenReview.
  arXiv  Detail & Related papers  (2025-05-12T16:02:52Z)
• Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checking [11.300523252168327]
  Evtextsuperscript2R combines the strengths of reference-based evaluation and verdict-level proxy scoring.<n>Evtextsuperscript2R consistently outperforms existing scoring approaches in accuracy and robustness.
  arXiv  Detail & Related papers  (2024-11-08T07:05:06Z)
• Large Language Models are not Fair Evaluators [60.27164804083752]
  We find that the quality ranking of candidate responses can be easily hacked by altering their order of appearance in the context. This manipulation allows us to skew the evaluation result, making one model appear considerably superior to the other. We propose a framework with three simple yet effective strategies to mitigate this issue.
  arXiv  Detail & Related papers  (2023-05-29T07:41:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.