An Exploratory Study of Agent Plans for Agentic AI Coding Tools in Open-Source Software

• URL: http://arxiv.org/abs/2608.04661v2
• Date: Thu, 06 Aug 2026 13:23:10 GMT
• Title: An Exploratory Study of Agent Plans for Agentic AI Coding Tools in Open-Source Software
• Abstract summary: Repository-level configuration artifacts allow developers to provide guidance for agentic AI coding tools, such as Claude Code, Gemini, etc.<n>We present an exploratory study of Agent Plans in open-source software repositories, examining how plan files are preserved, which development activities they support, and what information they provide to guide agent execution.
• Score: 16.01774662182876
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Repository-level configuration artifacts allow developers to provide guidance for agentic AI coding tools, such as Claude Code, Gemini, etc. Although prior research has examined repository-shared context files that capture project-level instructions and conventions (e.g., AGENTS.md files), little is known about more task-oriented artifacts such as Agent Plans. We present an exploratory study of Agent Plans in open-source software repositories, examining how plan files are preserved, which development activities they support, and what information they provide to guide agent execution. We screened 36,710 GitHub repositories belonging to engineered software projects and identified 85 Markdown plan files from 10 repositories. Within this highly concentrated corpus, Agent Plans supported several kinds of software engineering work, including maintenance, design, construction, quality-related work, and process support. They also provided task-oriented execution guidance, most commonly through implementation steps, concrete files and locations, and testing and validation information. Overall, repository-preserved Agent Plans under these tool-specific directories appear to be a narrow but informative artifact for studying task intent and execution guidance in human-agent workflows.

Related papers

• AgentLogs: A Dataset for Opening the Black Box of GitHub's Cloud Agent [0.8644909837301148]
  AgentLogs is a large-scale dataset of agent activity on GitHub.<n>It comprises 307,416 agent tasks and 549,239 agent sessions in 35,810 of the 1,812,362 popular public repositories.
  arXiv  Detail & Related papers  (2026-08-29T11:38:19Z)
• ToolRosetta: Bridging Open-Source Repositories and Large Language Model Agents through Automated Tool Standardization [51.92237664440418]
  ToolRosetta is a framework that automatically translates open-source code repositories and APIs into MCP-compatible tools.<n>Given a user task, ToolRosetta autonomously plans toolchains, identifies relevant toolchains, and converts them into executable MCP services.
  arXiv  Detail & Related papers  (2026-03-10T07:19:43Z)
• RepoLaunch: Automating Build&Test Pipeline of Code Repositories on ANY Language and ANY Platform [49.43594274832262]
  We introduce RepoLaunch, the first agent capable of automatically resolving dependencies, compiling source code, and extracting test results for repositories across arbitrary programming languages and operating systems.<n>RepoLaunch automates the remaining steps, enabling scalable benchmarking and training of coding agents and LLMs.
  arXiv  Detail & Related papers  (2026-03-05T10:15:13Z)
• OpenSage: Self-programming Agent Generation Engine [56.399761469404496]
  We propose OpenSage, the first agent development kit (ADK) to automatically create agents with self-generated topology and toolsets.<n>OpenSage offers effective functionality for agents to create and manage their own sub-agents and toolkits.<n>We believe OpenSage can pave the way for the next generation of agent development, shifting the focus from human-centered to AI-centered paradigms.
  arXiv  Detail & Related papers  (2026-02-18T21:16:29Z)
• Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents? [3.2610504259514754]
  We study whether context files are effective for real-world tasks.<n>We find that context files tend to reduce task success rates compared to providing no repository context.<n>We conclude that unnecessary requirements from context files make tasks harder, and human-written context files should describe only minimal requirements.
  arXiv  Detail & Related papers  (2026-02-12T14:15:22Z)
• OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding [57.39403818250357]
  We introduce OctoBench, which benchmarks scaffold-aware instruction following in repository-grounded agentic coding.<n> OctoBench includes 34 environments and 217 tasks instantiated under three scaffold types, and is paired with 7,098 objective checklist items.<n>Experiments reveal a systematic gap between task-solving and scaffold-aware compliance, underscoring the need for training and evaluation.
  arXiv  Detail & Related papers  (2026-01-15T12:36:08Z)
• Agent READMEs: An Empirical Study of Context Files for Agentic Coding [8.019313057979522]
  We study 2,303 agent context files from 1,925 repositories to characterize their structure, maintenance, and content.<n>We find that these files are not static documentation but complex, difficult-to-read artifacts that evolve like configuration code, maintained through frequent, small additions.<n>These findings indicate that while developers use context files to make agents functional, they provide few guardrails to ensure that agent-written code is secure or performant, highlighting the need for improved tooling and practices.
  arXiv  Detail & Related papers  (2025-11-17T02:18:55Z)
• Context Engineering for AI Agents in Open-Source Software [13.236926479239754]
  GenAI-based coding assistants have disrupted software development.<n>Their next generation is agent-based, operating with more autonomy and potentially without human oversight.<n>One challenge is to provide AI agents with sufficient context about the software projects they operate in.
  arXiv  Detail & Related papers  (2025-10-24T12:55:48Z)
• OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use [101.57043903478257]
  The dream to create AI assistants as capable and versatile as the fictional J.A.R.V.I.S from Iron Man has long captivated imaginations.<n>With the evolution of (multi-modal) large language models ((M)LLMs), this dream is closer to reality.<n>This survey aims to consolidate the state of OS Agents research, providing insights to guide both academic inquiry and industrial development.
  arXiv  Detail & Related papers  (2025-08-06T14:33:45Z)
• Beyond pip install: Evaluating LLM Agents for the Automated Installation of Python Projects [11.418182511485032]
  Large Language Model (LLM) based agents have been proposed for performing repository level' tasks.<n>We argue that one important task is missing, which is to fulfil project level dependency by installing other repositories.<n>We introduce a benchmark of repository installation tasks curated from 40 open source Python projects, which includes a ground truth installation process for each target repository.<n>Experiments reveal that 55% of the studied repositories can be automatically installed by our agent at least one out of ten times.
  arXiv  Detail & Related papers  (2024-12-09T08:37:06Z)
• Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration [64.19431011897515]
  This paper presents Alibaba LingmaAgent, a novel Automated Software Engineering method designed to comprehensively understand and utilize whole software repositories for issue resolution.<n>Our approach introduces a top-down method to condense critical repository information into a knowledge graph, reducing complexity, and employs a Monte Carlo tree search based strategy.<n>In production deployment and evaluation at Alibaba Cloud, LingmaAgent automatically resolved 16.9% of in-house issues faced by development engineers, and solved 43.3% of problems after manual intervention.
  arXiv  Detail & Related papers  (2024-06-03T15:20:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.