DeNovoSWE: Scaling Long-Horizon Environments for Generating Entire Repositories from Scratch

• URL: http://arxiv.org/abs/2606.10728v2
• Date: Tue, 16 Jun 2026 03:49:03 GMT
• Title: DeNovoSWE: Scaling Long-Horizon Environments for Generating Entire Repositories from Scratch
• Abstract summary: We introduce textbfDeNovoSWE, a large-scale dataset for whole-repository generation.<n>DeNovoSWE comprises 4,818 high-quality instances, where each instance requires generating a complete repository from documentation.<n>Fine-tuning Qwen3-30B-A3B on DeNovoSWE substantially improves long-horizon SWE performance, raising its score on the challenging BeyondSWE-Doc2Repo benchmark from 5.8% to 47.2%.
• Score: 89.87798465720181
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As the capabilities of LLM-based code agents continue to advance, their expected role is expanding beyond localized bug fixing in existing codebases toward architecting and implementing complete software repositories from high-level specifications. However, training agents for such long-horizon software engineering tasks remains difficult due to the scarcity of large-scale, verifiable whole-repository generation data. In this paper, we introduce \textbf{DeNovoSWE}, a large-scale dataset for whole-repository generation. DeNovoSWE comprises 4,818 high-quality instances, where each instance requires generating a complete repository from documentation. Our dataset is automatically constructed through a carefully designed sandboxed agentic workflow, enabling scalable curation without human annotation. DeNovoSWE is constructed with "divide and conquer" and critic-repair philosophy. To balance data quality and diversity, we further introduce a difficulty-aware trajectory filtering strategy. Fine-tuning Qwen3-30B-A3B on DeNovoSWE substantially improves long-horizon SWE performance, raising its score on the challenging BeyondSWE-Doc2Repo benchmark from 5.8% to 47.2%.

Related papers

• KAT-Coder-V2.5 Technical Report [61.907486544595336]
  KAT-Coder-V2.5 is a coding-focused agentic model trained to act autonomously inside real, executable repositories.<n>It reconstructs multilingual repositories into sandboxed environments with fail-to-pass and pass-to-pass verification at scale.<n>We further scale reinforcement learning with harness randomization, a reliability-hardened sandbox, an asymmetric actor--critic PPO with hindsight-augmented value estimation, and a harness-oriented reward framework.
  arXiv  Detail & Related papers  (2026-07-06T08:14:02Z)
• BeyondSWE: Can Current Code Agent Survive Beyond Single-Repo Bug Fixing? [61.247730037229815]
  We introduce BeyondSWE, a comprehensive benchmark that broadens existing evaluations along two axes - resolution scope and knowledge scope.<n>To investigate the role of external knowledge, we develop SearchSWE, a framework that integrates deep search with coding abilities.<n>This work offers both a realistic, challenging evaluation benchmark and a flexible framework to advance research toward more capable code agents.
  arXiv  Detail & Related papers  (2026-03-03T17:52:01Z)
• Immersion in the GitHub Universe: Scaling Coding Agents to Mastery [60.359983359258955]
  ScaleSWE is an automated, sandboxed multi agent workflow designed to construct high quality SWE data at scale.<n>The system coordinates three specialized agents for environment setup, test creation, and problem description synthesis to process 6 million pull requests across 5200 repositories.
  arXiv  Detail & Related papers  (2026-02-10T15:30:19Z)
• NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents [79.29376673236142]
  Existing benchmarks fail to rigorously evaluate the long-horizon capabilities required to build complete software systems.<n>We present NL2Repo Bench, a benchmark explicitly designed to evaluate the long-horizon repository generation ability of coding agents.
  arXiv  Detail & Related papers  (2025-12-14T15:12:13Z)
• Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling [7.753074942497876]
  We introduce CodeProjectEval, a project-level code generation dataset built from 18 real-world repositories with 12.7 files and 2,388.6 lines of code per task.<n>We propose ProjectGen, a multi-agent framework that decomposes projects into architecture design, skeleton generation, and code filling stages.<n>Experiments show that ProjectGen achieves state-of-the-art performance, passing 52/124 test cases on the small-scale project-level code generation dataset DevBench.
  arXiv  Detail & Related papers  (2025-11-05T12:12:35Z)
• PlotCraft: Pushing the Limits of LLMs for Complex and Interactive Data Visualization [82.96200364977737]
  We introduce PlotCraft, a new benchmark featuring 1k challenging visualization tasks.<n>PlotCraft is structured around seven high-level visualization tasks and encompasses 48 distinct chart types.<n>It is the first to systematically evaluate both single-turn generation and multi-turn refinement across a diverse spectrum of task complexities.
  arXiv  Detail & Related papers  (2025-10-15T10:14:39Z)
• SWE-Mirror: Scaling Issue-Resolving Datasets by Mirroring Issues Across Repositories [15.458389392000706]
  SWE-Mirror is a pipeline that distills a real-world issue's semantic essence, mirrors it into another repository with a configured Gym environment, and re-animates it as a verifiable issue-resolving task.<n>Applying SWE-Mirror to 40 repositories across 4 languages, we have curated a dataset with 60,671 issue-resolving tasks.<n>Post-training experiments show that models trained with the dataset exhibit improvements in issue-resolving capabilities.
  arXiv  Detail & Related papers  (2025-09-10T16:15:23Z)
• Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs [19.766885088032932]
  Software engineering (SWE) has emerged as a crucial testbed for next-generation LLM agents.<n>Most existing datasets are limited to only a few thousand GitHub-sourced instances.<n>We propose an incremental, automated data-curation pipeline that systematically scales both the volume and diversity of SWE datasets.
  arXiv  Detail & Related papers  (2025-06-24T03:53:36Z)
• Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Texts [65.84370471189676]
  We look at large-scale intermediate pre-training of decomposition-based transformers using distant supervision from comparable texts. We show that with such intermediate pre-training, developing robust decomposition-based models for a diverse range of tasks becomes more feasible.
  arXiv  Detail & Related papers  (2022-10-30T15:38:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.