ReproAgent: Contract-Guided Paper-to-Code Reproduction

• URL: http://arxiv.org/abs/2608.24291v1
• Date: Tue, 25 Aug 2026 09:19:00 GMT
• Title: ReproAgent: Contract-Guided Paper-to-Code Reproduction
• Abstract summary: Paper-to-code reproduction asks scientific AI agents to turn research papers into executable repositories that preserve the paper's method, protocol and artifacts.<n>We introduce ReproAgent, a four-stage Prepare--Plan--Repair pipeline built around a persistent implementation contract with two channels.<n>On PaperBench Code-Dev, ReproAgent reaches the highest mean score among same-backbone scaffolds under both Claude-Sonnet-4.5 and Gemini-3-Flash.
• Score: 19.371901718319876
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Paper-to-code reproduction asks scientific AI agents to turn research papers into executable repositories that preserve the paper's method, protocol and artifacts. This is difficult because the specification is split: explicit paper content such as algorithms, metrics and artifacts is often lost across long agent trajectories, while implicit details such as framework defaults and conventions inherited from related work are absent from the paper. We introduce ReproAgent, a four-stage Prepare--Plan--Generate--Repair pipeline built around a persistent implementation contract with two channels: an implementation-requirement channel that turns paper snippets into code obligations, and a reference-evidence channel that retrieves content and structure evidence from related repositories. Both are bound to work packages, projected into file-level contracts, and consumed across generation and repair. On PaperBench Code-Dev, ReproAgent reaches the highest mean score among same-backbone scaffolds under both Claude-Sonnet-4.5 and Gemini-3-Flash. End-to-end channel ablations and per-paper cases support the contribution of both channels. Code and experimental artifacts are publicly available.

Related papers

• PaperCompiler: Faithful Paper-to-Code Generation via Repository-Level Specification Compilation [24.326718323836293]
  We introduce PaperCompiler, a paper-to-code generation framework that compiles paper-grounded evidence into explicit repository-level implementation specifications.<n>The resulting specifications encode non-degradation requirements, ownership assignments, cross-file dependencies, and file-level constraints.<n>PaperCompiler outperforms strong baselines on Paper2CodeBench, achieving a 13.8% relative improvement in reference-based fidelity.
  arXiv  Detail & Related papers  (2026-09-02T08:21:03Z)
• Fetch-then-Explore: Decoupling Selection from Extraction over a Persistent Workspace for Search Agents [60.97609995940902]
  We propose textbfFetch-then-Explore for search agents.<n>It separates page selection from evidence extraction and keeps what it selects.<n>It leads BrowseComp accuracy at every backbone and generally matches or exceeds the baselines on WideSearch.
  arXiv  Detail & Related papers  (2026-08-03T11:58:21Z)
• Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction [57.138342889101345]
  Tencent WorkBuddy Bench is a multi-domain evaluation suite for coding agents.<n>This report documents its construction methodology, scoring protocol, and a cross-model leaderboard.
  arXiv  Detail & Related papers  (2026-07-23T04:34:06Z)
• ProjAgent: Procedural Similarity Retrieval for Repository-Level Code Generation [6.0914572974317]
  ProjAgent is a repository-level code generation system that introduces procedural similarity as an explicit retrieval signal.<n>ProjAgent achieves 41.14% Pass@1, outperforming existing retrieval-based baselines.
  arXiv  Detail & Related papers  (2026-07-09T16:50:54Z)
• Remember Your Trace: Memory-Guided Long-Horizon Agentic Framework for Consistent and Hierarchical Repository-Level Code Documentation [27.40384564559557]
  We propose MemDocAgent, a framework that generates documentation within a single, integrated context spanning the entire repository.<n>MemDocAgent achieves the best performance over both open and closed-source baselines.
  arXiv  Detail & Related papers  (2026-05-14T08:35:20Z)
• HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Execution [41.22400989548889]
  Hierarchical Research Agent System (HiRAS) is a hierarchical multi-agent framework for end-to-end experiment reproduction.<n>HiRAS employs supervisory manager agents to coordinate specialised agents across fine-grained stages.
  arXiv  Detail & Related papers  (2026-04-20T02:59:43Z)
• What Papers Don't Tell You: Recovering Tacit Knowledge for Automated Paper Reproduction [57.86097956633207]
  method is a graph-based agent framework for generating executable code from academic papers.<n>On an extended ReproduceBench spanning 3 domains, 10 tasks, and 40 recent papers, method achieves an average performance gap of 10.04% against official implementations.
  arXiv  Detail & Related papers  (2026-03-02T12:33:31Z)
• Model-Document Protocol for AI Search [11.377241012645994]
  We introduce the Model-Document Protocol (MDP), a general framework that formalizes how raw text is bridged to large language models (LLMs)<n>Rather than treating retrieval as passage fetching, MDP defines multiple pathways that transform unstructured documents into task-specific, LLM-ready inputs.<n>As an instantiation, we present MDP-Agent, which realizes the protocol through an agentic process.
  arXiv  Detail & Related papers  (2025-10-29T04:29:17Z)
• Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning [70.04746094652653]
  We introduce PaperCoder, a framework that transforms machine learning papers into functional code repositories.<n>PaperCoder operates in three stages: planning, designs the system architecture with diagrams, identifies file dependencies, and generates configuration files.<n>We then evaluate PaperCoder on generating code implementations from machine learning papers based on both model-based and human evaluations.
  arXiv  Detail & Related papers  (2025-04-24T01:57:01Z)
• Contextual Document Embeddings [77.22328616983417]
  We propose two complementary methods for contextualized document embeddings. First, an alternative contrastive learning objective that explicitly incorporates the document neighbors into the intra-batch contextual loss. Second, a new contextual architecture that explicitly encodes neighbor document information into the encoded representation.
  arXiv  Detail & Related papers  (2024-10-03T14:33:34Z)
• GERE: Generative Evidence Retrieval for Fact Verification [57.78768817972026]
  We propose GERE, the first system that retrieves evidences in a generative fashion. The experimental results on the FEVER dataset show that GERE achieves significant improvements over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-04-12T03:49:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.