Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments

• URL: http://arxiv.org/abs/2607.28591v1
• Date: Thu, 30 Jul 2026 17:44:31 GMT
• Title: Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments
• Abstract summary: Change2Task converts merged pull requests into verified tasks on healthy modern revisions of the same repository.<n>It aligns historical evidence with evolved code, reconstructs task states through Patch Reversal, Code Mapping, or Agent Reconstruction.<n>We evaluate the system through five common and widely adopted coding agent task families: Bug Fix, Feature Addition, Test Generation, Application Programming Interface Migration, and Security Repair.
• Score: 46.958911818255565
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation. Each task must couple a realistic software state with a specification, development tools, and reliable verification. To expand this supply, we present Change2Task, a system grounded in repository history that converts merged pull requests into verified tasks on healthy modern revisions of the same repository. It aligns historical evidence with evolved code, reconstructs task states through Patch Reversal, Code Mapping, or Agent Reconstruction, and validates the lifecycle from a healthy base to a task state and a restored state. By deriving multiple tasks grounded in developer evidence from maintained environments, Change2Task provides executable data for coding agent training and evaluation while reducing repeated environment setup, storage, and task construction effort. We evaluate the system through five common and widely adopted coding agent task families: Bug Fix, Feature Addition, Test Generation, Application Programming Interface Migration, and Security Repair. Starting from 1,130 source changes eligible for construction, Change2Task achieves 79.6% verified task construction success across these task families. On a matched candidate set, it recovers 29.2% more verified tasks than a construction baseline based on pull requests. Historical and reconstructed cases achieve up to 98.0% matched outcome agreement under agent evaluation, while reuse of modern bases reduces measured expenditure across the complete pipeline by 10.8%.

Related papers

• Grounded Skill Synthesis from Code at Scale for Agentic Intelligence [22.319997851901714]
  Existing methods face two limitations: trajectory-based synthesis requires interactions with specific environments, while document-derived skills may lack executable evidence and verification.<n>We present Code2Skill, a fully automated pipeline that transforms selected code units into implementation-anchored records of atomic operations, composite repositories, and recurring patterns.<n>Code2Skill produces CodeSkillBank, a grounded bank of 1,006,822 accepted records with workflow, boundary, provenance, and source-evidence metadata.
  arXiv  Detail & Related papers  (2026-09-04T03:37:06Z)
• Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction [57.138342889101345]
  Tencent WorkBuddy Bench is a multi-domain evaluation suite for coding agents.<n>This report documents its construction methodology, scoring protocol, and a cross-model leaderboard.
  arXiv  Detail & Related papers  (2026-07-23T04:34:06Z)
• RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations [51.43574078961796]
  We introduce RepoMirage, a two-stage evaluation suite built on SWE-Bench Verified.<n>RepoMirage-Perturb applies three types of semantics-preserving repository-level perturbations, revealing a clear performance drop when correct solving requires broader context access.<n>We propose RepoAnchor, a structure-first prototype workflow that separates repository exploration from downstream problem solving.
  arXiv  Detail & Related papers  (2026-05-25T06:26:43Z)
• Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows [67.92316850084575]
  We introduce Claw-Eval-Live, a live benchmark for workflow agents.<n>Each release is constructed from public workflow-demand signals.<n>Claw-Eval-Live records execution traces, audit logs, service state, and post-run workspace artifacts.
  arXiv  Detail & Related papers  (2026-04-30T17:23:19Z)
• Beyond Isolated Tasks: A Framework for Evaluating Coding Agents on Sequential Software Evolution [5.10403054516716]
  Existing datasets evaluate performance on isolated, single pull request (PR) tasks in a stateless manner.<n>We introduce an automated coding task generation framework, which helps generate our dataset SWE-STEPS.
  arXiv  Detail & Related papers  (2026-04-03T13:44:40Z)
• SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks [55.76734816061826]
  We introduce SlopCodeBench, a language-agnostic benchmark comprising 20 problems and 93 checkpoints.<n>We track two trajectory-level quality signals: verbosity, the fraction of redundant or duplicated code, and structural erosion.<n>No agent solves any problem end-to-end across 11 models; the highest checkpoint solve rate is 17.2%.
  arXiv  Detail & Related papers  (2026-03-25T19:26:44Z)
• FeatureBench: Benchmarking Agentic Coding for Complex Feature Development [42.26354337364403]
  FeatureBench is a benchmark designed to evaluate agentic coding performance in end-to-end, feature-oriented software development.<n>It incorporates an execution-based evaluation protocol and a scalable test-driven method that automatically derives tasks from code repositories with minimal human effort.<n> Empirical evaluation reveals that the state-of-the-art agentic model, such as Claude 4.5 Opus, achieves a 74.4% resolved rate on SWE-bench.
  arXiv  Detail & Related papers  (2026-02-11T16:06:32Z)
• GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging [41.754784344572286]
  We release GitTaskBench, a benchmark for evaluating code agents in real-world scenarios.<n>Each task pairs a relevant repository with an automated, human-curated evaluation harness.<n>We also propose the alpha-value metric to quantify the economic benefit of agent performance.
  arXiv  Detail & Related papers  (2025-08-26T12:48:05Z)
• RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation [96.75695811963242]
  RepoCoder is a framework to streamline the repository-level code completion process. It incorporates a similarity-based retriever and a pre-trained code language model. It consistently outperforms the vanilla retrieval-augmented code completion approach.
  arXiv  Detail & Related papers  (2023-03-22T13:54:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.