Beyond Paper-to-Paper: Structured Profiling and Rubric Scoring for Paper-Reviewer Matching

• URL: http://arxiv.org/abs/2604.05866v1
• Date: Tue, 07 Apr 2026 13:27:40 GMT
• Title: Beyond Paper-to-Paper: Structured Profiling and Rubric Scoring for Paper-Reviewer Matching
• Abstract summary: P2R is a training-free framework that shifts from implicit paper-to-paper matching to explicit profile-based matching.<n>It first performs hybrid retrieval that combines semantic and aspect-level signals to form a high-recall candidate pool.<n>Experiments on NeurIPS, SIGIR, and SciRepEval show that P2R consistently outperforms state-of-the-art baselines.
• Score: 5.731351860196096
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As conference submission volumes continue to grow, accurately recommending suitable reviewers has become a challenge. Most existing methods follow a ``Paper-to-Paper'' matching paradigm, implicitly representing a reviewer by their publication history. However, effective reviewer matching requires capturing multi-dimensional expertise, and textual similarity to past papers alone is often insufficient. To address this gap, we propose P2R, a training-free framework that shifts from implicit paper-to-paper matching to explicit profile-based matching. P2R uses general-purpose LLMs to construct structured profiles for both submissions and reviewers, disentangling them into Topics, Methodologies, and Applications. Building on these profiles, P2R adopts a coarse-to-fine pipeline to balance efficiency and depth. It first performs hybrid retrieval that combines semantic and aspect-level signals to form a high-recall candidate pool, and then applies an LLM-based committee to evaluate candidates under strict rubrics, integrating both multi-dimensional expert views and a holistic Area Chair perspective. Experiments on NeurIPS, SIGIR, and SciRepEval show that P2R consistently outperforms state-of-the-art baselines. Ablation studies further verify the necessity of each component. Overall, P2R highlights the value of explicit, structured expertise modeling and offers practical guidance for applying LLMs to reviewer matching.

Related papers

• SSR-GRPO: Integrating Supervision and Semantic IDs into Reinforcement Learning for Dense Retrieval in E-commerce [4.188658309210585]
  Embedding-based retrieval (EBR) is pivotal in e-commerce search but often struggles with complex semantics.<n>Recent methods often fine-tune large language models (LLMs) for representation learning.<n>We propose Supervised Retrieval-GRPO with Semantic Identifiers (SSR-GRPO)
  arXiv  Detail & Related papers  (2026-08-20T03:19:20Z)
• Training Documents Reranker with Search Rubrics for Deep Research Agent [54.08506174105984]
  We propose search-oriented rubrics that textitexplicitly define the requirements that high-quality document sets should satisfy.<n>We further train a document reranker textbfRubricRanker to select a high-quality subset from retrieved documents.
  arXiv  Detail & Related papers  (2026-08-04T12:11:14Z)
• Can LLMs Write Reliable Rubrics? A Meta-Evaluation for Experiment Reproduction [38.170020271877085]
  We present the first systematic meta-evaluation of LLM-generated rubrics for paper reproduction.<n>We reformulate rubrics into a checklist-style format and evaluate four generation settings across two backbone models.
  arXiv  Detail & Related papers  (2026-07-14T14:52:42Z)
• Align Documents to Questions: Question-Oriented Document Rewriting for Retrieval-Augmented Generation [51.55755193937205]
  We propose QREAM, a style-controlled rewriter that aligns retrieved documents with a question-oriented style while preserving facts.<n>Our framework consists of two stages: (1) QREAM-ICL, which uses stylistic seeds to guide iterative rewriting exploration; and (2) QREAM-FT, a lightweight student model distilled from denoised ICL outputs.
  arXiv  Detail & Related papers  (2026-04-19T08:39:21Z)
• ReviewGrounder: Improving Review Substantiveness with Rubric-Guided, Tool-Integrated Agents [50.27474750319121]
  Review text according to paper-specific rubrics derived from official guidelines, the paper's content, and human-written reviews.<n>We introduce REVIEWBENCH, a benchmark evaluating review text according to paper-specific rubrics derived from official guidelines, the paper's content, and human-written reviews.<n>We propose REVIEWGROUNDER, a rubric-guided, tool-integrated multi-agent framework that decomposes reviewing into drafting and grounding stages.
  arXiv  Detail & Related papers  (2026-04-15T16:33:04Z)
• PaperGuide: Making Small Language-Model Paper-Reading Agents More Efficient [20.72001543887772]
  Recent progress in large language models (LLMs) has spurred interest in autonomous agents that can read scientific papers and extract task-relevant information.<n>Most existing approaches rely either on heavily engineered prompting or on a conventional SFT-RL training pipeline.<n>We propose Paper RL, a framework that mitigates these issues by separating high-level planning from fine-grained execution.
  arXiv  Detail & Related papers  (2026-01-19T12:07:51Z)
• OpenNovelty: An LLM-powered Agentic System for Verifiable Scholarly Novelty Assessment [63.662126457336534]
  OpenNovelty is an agentic system for transparent, evidence-based novelty analysis.<n>It grounds all assessments in retrieved real papers, ensuring verifiable judgments.<n>OpenNovelty aims to empower the research community with a scalable tool that promotes fair, consistent, and evidence-backed peer review.
  arXiv  Detail & Related papers  (2026-01-04T15:48:51Z)
• Chain of Retrieval: Multi-Aspect Iterative Search Expansion and Post-Order Search Aggregation for Full Paper Retrieval [68.71038700559195]
  Chain of Retrieval(COR) is a novel iterative framework for full-paper retrieval.<n>We present SCIBENCH, a benchmark providing both complete and segmented contexts of full papers for queries and candidates.
  arXiv  Detail & Related papers  (2025-07-14T08:41:53Z)
• Can LLMs Generate Tabular Summaries of Science Papers? Rethinking the Evaluation Protocol [83.90769864167301]
  Literature review tables are essential for summarizing and comparing collections of scientific papers.<n>We explore the task of generating tables that best fulfill a user's informational needs given a collection of scientific papers.<n>Our contributions focus on three key challenges encountered in real-world use: (i) User prompts are often under-specified; (ii) Retrieved candidate papers frequently contain irrelevant content; and (iii) Task evaluation should move beyond shallow text similarity techniques.
  arXiv  Detail & Related papers  (2025-04-14T14:52:28Z)
• Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquiry [0.0]
  Deep learning's preponderance across scientific domains has reshaped high-stakes decision-making.<n>This paper examines the complexities of combining Right-to-Privacy (RTP) and Right-to-Explanation (RTE)
  arXiv  Detail & Related papers  (2024-12-30T08:43:28Z)
• LitLLMs, LLMs for Literature Review: Are we there yet? [15.785989492351684]
  This paper explores the zero-shot abilities of recent Large Language Models in assisting with the writing of literature reviews based on an abstract.<n>For retrieval, we introduce a novel two-step search strategy that first uses an LLM to extract meaningful keywords from the abstract of a paper.<n>In the generation phase, we propose a two-step approach that first outlines a plan for the review and then executes steps in the plan to generate the actual review.
  arXiv  Detail & Related papers  (2024-12-15T01:12:26Z)
• Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions [62.0123588983514]
  Large Language Models (LLMs) have demonstrated wide-ranging applications across various fields. We reformulate the peer-review process as a multi-turn, long-context dialogue, incorporating distinct roles for authors, reviewers, and decision makers. We construct a comprehensive dataset containing over 26,841 papers with 92,017 reviews collected from multiple sources.
  arXiv  Detail & Related papers  (2024-06-09T08:24:17Z)
• Relative Preference Optimization: Enhancing LLM Alignment through Contrasting Responses across Identical and Diverse Prompts [95.09994361995389]
  Relative Preference Optimization (RPO) is designed to discern between more and less preferred responses derived from both identical and related prompts. RPO has demonstrated a superior ability to align large language models with user preferences and to improve their adaptability during the training process.
  arXiv  Detail & Related papers  (2024-02-12T22:47:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.