Judgment-Grounded Expansion for Peer Review Generation

• URL: http://arxiv.org/abs/2606.23233v1
• Date: Mon, 22 Jun 2026 12:20:11 GMT
• Title: Judgment-Grounded Expansion for Peer Review Generation
• Abstract summary: We formalize judgment-grounded expansion, a human-AI collaboration mode where a reviewer provides an evaluative claim and the system expands it into review comment candidate(s)<n>We study two practical challenges for judgment-grounded expansion: scalable evaluation and candidate set curation.<n>Our work establishes judgment-grounded expansion as a concrete task and provides empirical and methodological foundations for the design of future collaborative review generation systems.
• Score: 68.04679314462481
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic review generation is a promising direction for accelerating scientific progress. While most work adopts an end-to-end setup, its fully automated nature makes it less suitable for settings that demand accountability. To better balance automation and accountability, we formalize judgment-grounded expansion, a human-AI collaboration mode where a reviewer provides an evaluative claim and the system expands it into review comment candidate(s). We model it as a structured generate-check-refine process and conduct a user study to collect human-model interaction data. We study two practical challenges for judgment-grounded expansion: scalable evaluation and candidate set curation. We develop methods to simulate the process for large-scale evaluation, and show that conformal prediction is well suited to balancing candidate set size and target coverage. Our work establishes judgment-grounded expansion as a concrete task and provides empirical and methodological foundations for the design of future collaborative review generation systems.

Related papers

• Reality Is the Final Verifier: On Two Key Gaps in Agentic Software Engineering [78.51705689800838]
  We propose an assurance-revision loop that uses deployment evidence to revise the requirements, model, or evaluator.<n>We then cast assured agentic development as a resource-allocation problem over human judgment, agent capability, and compute.
  arXiv  Detail & Related papers  (2026-09-10T17:58:48Z)
• Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future [50.89292049215399]
  Large language models (LLMs) have motivated methods that assist or automate different stages of the peer review process.<n>We synthesize techniques for (i) peer review generation, including fine-tuning strategies, agent-based systems, RL-based methods, and emerging paradigms to enhance generation.<n>We catalog datasets, compare modeling choices, and discuss limitations, ethical concerns, and future directions.
  arXiv  Detail & Related papers  (2026-04-30T14:28:21Z)
• Pitfalls in Evaluating Interpretability Agents [91.49742416116635]
  We build an agentic system in which a research agent iteratively designs experiments and refines hypotheses.<n>Our work demonstrates fundamental challenges in evaluating complex automated interpretability systems.
  arXiv  Detail & Related papers  (2026-03-20T16:27:17Z)
• PaperRepro: Automated Computational Reproducibility Assessment for Social Science Papers [33.12402746591649]
  PaperRepro is a novel two-stage, multi-agent approach for automated assessment.<n>In the execution stage, agents execute the reproduction package and edit the code to capture reproduced results as explicit artifacts.<n>In the evaluation stage, agents evaluate agents using explicit evidence.
  arXiv  Detail & Related papers  (2026-02-10T09:04:59Z)
• Beyond Static Evaluation: Rethinking the Assessment of Personalized Agent Adaptability in Information Retrieval [12.058221341033835]
  We propose a conceptual lens for rethinking evaluation in adaptive personalization.<n>We organize this lens around three core components: (1) persona-based user simulation with temporally evolving preference models; (2) structured elicitation protocols inspired by reference interviews to extract preferences in context; and (3) adaptation-aware evaluation mechanisms that measure how agent behavior improves across sessions and tasks.
  arXiv  Detail & Related papers  (2025-10-05T00:35:37Z)
• AI Agents-as-Judge: Automated Assessment of Accuracy, Consistency, Completeness and Clarity for Enterprise Documents [0.0]
  This study presents a modular, multi-agent system for the automated review of highly structured enterprise business documents using AI agents.<n>It uses modern orchestration tools such as LangChain, CrewAI, TruLens, and Guidance to enable section-by-section evaluation of documents.<n>It achieves 99% information consistency (vs. 92% for humans), halving error and bias rates, and reducing average review time from 30 to 2.5 minutes per document.
  arXiv  Detail & Related papers  (2025-06-23T17:46:15Z)
• Socio-Emotional Response Generation: A Human Evaluation Protocol for LLM-Based Conversational Systems [9.101091541480434]
  We propose a neural architecture that includes an intermediate step in planning socio-emotional strategies before response generation.<n>Our study shows that predicting a sequence of expected strategy labels and using this sequence to generate a response yields better results than a direct end-to-end generation scheme.
  arXiv  Detail & Related papers  (2024-11-26T08:15:36Z)
• Test-time Collective Prediction [73.74982509510961]
  Multiple parties in machine learning want to jointly make predictions on future test points. Agents wish to benefit from the collective expertise of the full set of agents, but may not be willing to release their data or model parameters. We explore a decentralized mechanism to make collective predictions at test time, leveraging each agent's pre-trained model.
  arXiv  Detail & Related papers  (2021-06-22T18:29:58Z)
• SummEval: Re-evaluating Summarization Evaluation [169.622515287256]
  We re-evaluate 14 automatic evaluation metrics in a comprehensive and consistent fashion. We benchmark 23 recent summarization models using the aforementioned automatic evaluation metrics. We assemble the largest collection of summaries generated by models trained on the CNN/DailyMail news dataset.
  arXiv  Detail & Related papers  (2020-07-24T16:25:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.