Repo2Run: Automated Building Executable Environment for Code Repository at Scale

• URL: http://arxiv.org/abs/2502.13681v3
• Date: Fri, 30 May 2025 06:25:20 GMT
• Title: Repo2Run: Automated Building Executable Environment for Code Repository at Scale
• Authors: Ruida Hu, Chao Peng, Xinchen Wang, Junjielong Xu, Cuiyun Gao,
• Abstract summary: We introduce Repo2Run, an agent aiming at automating the building of executable test environments for any repositories at scale.<n>Repo2Run iteratively builds the Docker image, runs unit tests based on the feedback of the building, and synthesizes the Dockerfile.<n>The resulting Dockerfile can then be used to create Docker container environments for running code and tests.
• Score: 8.795746370609855
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Scaling up executable code data is significant for improving language models' software engineering capability. The intricate nature of the process makes it labor-intensive, time-consuming and expert-knowledge-dependent to build a large number of executable code repositories, limiting the scalability of existing work based on running tests. The primary bottleneck lies in the automated building of test environments for different repositories, which is an essential yet underexplored task. To mitigate the gap, we introduce Repo2Run, the first LLM-based agent aiming at automating the building of executable test environments for any repositories at scale. Specifically, given a code repository, Repo2Run iteratively builds the Docker image, runs unit tests based on the feedback of the building, and synthesizes the Dockerfile until the entire pipeline is executed successfully. The resulting Dockerfile can then be used to create Docker container environments for running code and tests. We created a benchmark containing 420 Python repositories with unit tests for evaluation. The results illustrate that Repo2Run achieves an 86.0% success rate, outperforming SWE-agent by 77.0%. The resources of Repo2Run are available at https://github.com/bytedance/Repo2Run.

Related papers

• SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner [53.54568352375669]
  We introduce **SWE-Flow**, a novel data synthesis framework grounded in Test-Driven Development (TDD)<n>Unlike existing software engineering data that rely on human-submitted issues, **SWE-Flow** automatically infers incremental development steps directly from unit tests.<n>We generated 16,061 training instances and 2,020 test instances from real-world GitHub projects, creating the **SWE-Flow-Eval** benchmark.
  arXiv  Detail & Related papers  (2025-06-10T17:23:33Z)
• Compendium Manager: a tool for coordination of workflow management instances for bulk data processing in Python [0.0]
  Compendium Manager is a command-line tool written in Python to automate the provisioning, launch, and evaluation of bioinformatics pipelines.<n>It can gauge progress through a list of projects, load results into a shared database, and record detailed processing metrics for later evaluation and evaluation.
  arXiv  Detail & Related papers  (2025-05-16T15:49:40Z)
• Doctor: Optimizing Container Rebuild Efficiency by Instruction Re-Orchestration [11.027705516378875]
  We present Doctor, a method for improving Dockerfile build efficiency through instruction re-ordering. We developed a dependency taxonomy based on Dockerfile syntax and a historical modification analysis to prioritize frequently modified instructions. Experiments show Doctor improves 92.75% of Dockerfiles, reducing rebuild time by an average of 26.5%, with 12.82% of files achieving over a 50% reduction.
  arXiv  Detail & Related papers  (2025-04-02T13:53:35Z)
• EnvBench: A Benchmark for Automated Environment Setup [76.02998475135824]
  Large Language Models have enabled researchers to focus on practical repository-level tasks in software engineering domain. Existing studies on environment setup introduce innovative agentic strategies, but their evaluation is often based on small datasets. To address this gap, we introduce a comprehensive environment setup benchmark EnvBench.
  arXiv  Detail & Related papers  (2025-03-18T17:19:12Z)
• Automated Benchmark Generation for Repository-Level Coding Tasks [7.305342793164905]
  SetUpAgent is a fully automated system capable of historically accurate dependency setup, test execution, and result parsing.<n>We generate two new datasets: (i) SWEE-Bench an extended version of SWE-Bench encompassing hundreds of repositories, and (ii) SWA-Bench a benchmark focusing on applications rather than libraries.
  arXiv  Detail & Related papers  (2025-03-10T17:42:49Z)
• RepoST: Scalable Repository-Level Coding Environment Construction with Sandbox Testing [35.68087697258125]
  RepoST provides execution feedback for repository-level code generation for both training and evaluation.<n>RepoST-Train is a large-scale train set with 7,415 functions from 832 repositories.<n>We also build an evaluation dataset, RepoST-Eval, and benchmark 12 code generation models.
  arXiv  Detail & Related papers  (2025-03-10T14:16:08Z)
• Refactoring for Dockerfile Quality: A Dive into Developer Practices and Automation Potential [0.0]
  This paper explores the utility and practicality of automating Dockerfile using 600files from 358 open-source projects.<n>Our approach leads to an average reduction of 32% in image size and a 6% decrease in build duration, with improvements in understandability and maintainability observed in 77% and 91% of cases.
  arXiv  Detail & Related papers  (2025-01-23T23:10:47Z)
• ExecRepoBench: Multi-level Executable Code Completion Evaluation [45.963424627710765]
  We introduce a novel framework for enhancing code completion in software development through the creation of a repository-level benchmark ExecRepoBench.<n>We present a multi-level grammar-based completion methodology conditioned on the abstract syntax tree to mask code fragments at various logical units.<n>Then, we fine-tune the open-source LLM with 7B parameters on Repo-Instruct to produce a strong code completion baseline model Qwen2.5-Coder-Instruct-C.
  arXiv  Detail & Related papers  (2024-12-16T17:14:35Z)
• Beyond pip install: Evaluating LLM Agents for the Automated Installation of Python Projects [11.418182511485032]
  Large Language Model (LLM) based agents have been proposed for performing repository level' tasks.<n>We argue that one important task is missing, which is to fulfil project level dependency by installing other repositories.<n>We introduce a benchmark of repository installation tasks curated from 40 open source Python projects, which includes a ground truth installation process for each target repository.<n>Experiments reveal that 55% of the studied repositories can be automatically installed by our agent at least one out of ten times.
  arXiv  Detail & Related papers  (2024-12-09T08:37:06Z)
• Commit0: Library Generation from Scratch [77.38414688148006]
  Commit0 is a benchmark that challenges AI agents to write libraries from scratch.<n>Agents are provided with a specification document outlining the library's API as well as a suite of interactive unit tests.<n> Commit0 also offers an interactive environment where models receive static analysis and execution feedback on the code they generate.
  arXiv  Detail & Related papers  (2024-12-02T18:11:30Z)
• Codev-Bench: How Do LLMs Understand Developer-Centric Code Completion? [60.84912551069379]
  We present the Code-Development Benchmark (Codev-Bench), a fine-grained, real-world, repository-level, and developer-centric evaluation framework. Codev-Agent is an agent-based system that automates repository crawling, constructs execution environments, extracts dynamic calling chains from existing unit tests, and generates new test samples to avoid data leakage.
  arXiv  Detail & Related papers  (2024-10-02T09:11:10Z)
• CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents [49.68117560675367]
  Crab is the first benchmark framework designed to support cross-environment tasks. Our framework supports multiple devices and can be easily extended to any environment with a Python interface. The experimental results demonstrate that the single agent with GPT-4o achieves the best completion ratio of 38.01%.
  arXiv  Detail & Related papers  (2024-07-01T17:55:04Z)
• Arbitrarily Scalable Environment Generators via Neural Cellular Automata [55.150593161240444]
  We show that NCA environment generators maintain consistent, regularized patterns regardless of environment size. Our method scales a single-agent reinforcement learning policy to arbitrarily large environments with similar patterns.
  arXiv  Detail & Related papers  (2023-10-28T07:30:09Z)
• L2MAC: Large Language Model Automatic Computer for Extensive Code Generation [52.81694565226513]
  Transformer-based large language models (LLMs) are constrained by the fixed context window of the underlying transformer architecture. This paper presents L2MAC, the first practical LLM-based general-purpose stored-program automatic computer (von Neumann architecture) framework, for long and consistent output generation.
  arXiv  Detail & Related papers  (2023-10-02T16:55:19Z)
• RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation [96.75695811963242]
  RepoCoder is a framework to streamline the repository-level code completion process. It incorporates a similarity-based retriever and a pre-trained code language model. It consistently outperforms the vanilla retrieval-augmented code completion approach.
  arXiv  Detail & Related papers  (2023-03-22T13:54:46Z)
• EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Engine [69.47822647770542]
  parallel environment execution is often the slowest part of the whole system but receives little attention. With a curated design for paralleling RL environments, we have improved the RL environment simulation speed across different hardware setups. On a high-end machine, EnvPool achieves 1 million frames per second for the environment execution on Atari environments and 3 million frames per second on MuJoCo environments.
  arXiv  Detail & Related papers  (2022-06-21T17:36:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.