How Do Developers Maintain and Evolve Their Agents' Instructions? An Empirical Study

• URL: http://arxiv.org/abs/2606.25257v1
• Date: Wed, 24 Jun 2026 00:32:18 GMT
• Title: How Do Developers Maintain and Evolve Their Agents' Instructions? An Empirical Study
• Abstract summary: Agent Context Files (ACFs) have emerged as a practical mechanism to guide agents through structured instructions.<n>This paper plans to investigate the evolution of ACFs and their role in agent-driven development.
• Score: 9.708580060106414
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Context. Autonomous coding agents are increasingly used in software development, shifting parts of the engineering process to AI assistance. While this automation brings clear benefits, it introduces challenges in governance, traceability, and control over agent behavior. Agent Context Files (ACFs) have emerged as a practical mechanism to guide agents through structured instructions, yet little is known about how these artifacts are maintained and how their evolution relates to code development. Objective. This paper plans to investigate the evolution of ACFs and their role in agent-driven development. Specifically, we (1) classify ACF changes through a taxonomy grounded in software maintenance theory, (2) analyze how different types of changes are associated with code quality outcomes, and (3) examine their temporal patterns across the development lifecycle. Method. We conduct a large-scale mining study combining repositories with ACFs and agent-generated commits. We reconstruct ACF evolution at the commit level, classify changes using a qualitative approach, and analyze their association with code quality metrics. Statistical analyses and hypotheses are used to evaluate differences across maintenance categories, to inform future design of ACFs for governing autonomous coding agents.

Related papers

• Self-Evolving Coding Agents [11.43197414613246]
  Self-evolving coding agents can inspect repositories, invoke tools, execute tests, debug failures, and generate patches.<n>We first define self-evolving coding agents and distinguish them from conventional coding agents and general self-evolving agents.<n>We find that executable feedback, repository-level context, and coding trajectories give software engineering a distinctive role as a natural domain for agent self-evolution.
  arXiv  Detail & Related papers  (2026-08-04T09:43:46Z)
• How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests [5.6795075579540395]
  This study aims to characterize agentic pull requests (PR) in comparison to human generated PRs.<n>We first analyze how differences in merge rates between agentic and human generated PRs vary over time.<n>We then identify the types of development tasks where AI coding agents are predominantly applied.
  arXiv  Detail & Related papers  (2026-07-23T21:48:28Z)
• How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Study [14.012434371282382]
  This paper is the first to study how Software Engineering processes are changing in the development of SE agents.<n>We characterize a seven-stage workflow and a shift toward evaluation-driven development.<n>We identify six challenges that teams face, together with the practices they adopt to address them.
  arXiv  Detail & Related papers  (2026-07-12T17:40:36Z)
• Agentic Software: How AI Agents Are Restructuring the Software Paradigm [0.0]
  We argue that the emergence of AI agents constitutes a fundamental restructuring of what software is.<n>In the former, code is the carrier of pre-written decision logic; in the latter, the agent itself is the decision logic.<n>We conclude with a four-stage roadmap toward self-evolving agent ecosystems.
  arXiv  Detail & Related papers  (2026-06-04T02:30:06Z)
• From Prompt to Process: a Process Taxonomy and Comparative Assessment of Frameworks Supporting AI Software Development Agents [0.0]
  We focus on the operational frameworks that turn AI capabilities into process.<n>Each attack AI development through a different path: spec-driven development in full and lightweight variants, agent-driven agile planning, context engineering over the agent, worktree isolation and review, and recovery of operational specifications from legacy systems.<n>Our central contribution is a six-dimension process taxonomy: specification, context, roles, execution, validation and portability, with a scoring rubric that turns it into a replicable instrument.
  arXiv  Detail & Related papers  (2026-06-03T14:49:15Z)
• Code as Agent Harness [107.31925305395957]
  In emerging agentic systems, code is no longer only a target output.<n>Code increasingly serves as an operational substrate for agent reasoning, acting, environment modeling, and execution-based verification.<n>This survey provides a unified roadmap toward executable, verifiable, and stateful AI agent systems.
  arXiv  Detail & Related papers  (2026-05-18T17:59:03Z)
• How to Interpret Agent Behavior [56.59836196946289]
  We introduce ACT*ONOMY, a taxonomy for describing and analyzing agent behavior at runtime.<n>By providing a shared vocabulary, ACT*ONOMY helps researchers, agent designers, and end users interpret agent behavior more consistently.
  arXiv  Detail & Related papers  (2026-05-13T14:52:40Z)
• From Prompt-Response to Goal-Directed Systems: The Evolution of Agentic AI Software Architecture [0.0]
  Agentic AI denotes an architectural transition from stateless, prompt-driven generative models toward goal-directed systems.<n>This paper examines this transition by connecting intelligent agent theories, with contemporary LLM-centric approaches.<n>The study identifies a convergence toward standardized agent loops, registries, and auditable control mechanisms.
  arXiv  Detail & Related papers  (2026-02-11T03:34:48Z)
• An Empirical Study of Agent Developer Practices in AI Agent Frameworks [59.862193600499914]
  The rise of large language models (LLMs) has sparked a surge of interest in agents, leading to the rapid growth of agent frameworks.<n>Despite widespread use of agent frameworks, their practical applications and how they influence the agent development process remain underexplored.<n>More than 80% of developers report difficulties in identifying the frameworks that best meet their specific development requirements.
  arXiv  Detail & Related papers  (2025-12-01T17:52:15Z)
• Agentsway -- Software Development Methodology for AI Agents-based Teams [4.226647687395254]
  "Agentsway" is a novel software development framework designed for ecosystems where AI agents operate as first-class collaborators.<n>The framework defines distinct roles for planning, prompting, coding, testing, and fine-tuning agents.<n>Agentsway represents a foundational step toward the next generation of AI-native, self-improving software development methodologies.
  arXiv  Detail & Related papers  (2025-10-26T11:58:42Z)
• A Survey of Self-Evolving Agents: On Path to Artificial Super Intelligence [87.08051686357206]
  Large Language Models (LLMs) have demonstrated strong capabilities but remain fundamentally static.<n>As LLMs are increasingly deployed in open-ended, interactive environments, this static nature has become a critical bottleneck.<n>This survey provides the first systematic and comprehensive review of self-evolving agents.
  arXiv  Detail & Related papers  (2025-07-28T17:59:05Z)
• Understanding Software Engineering Agents Through the Lens of Traceability: An Empirical Study [15.97770416681533]
  Software engineering agents (SWE agents) operate autonomously by interpreting user input and responding to environmental feedback.<n>We present the first systematic study of SWE agent behavior through the lens of execution traces.
  arXiv  Detail & Related papers  (2025-06-10T00:41:54Z)
• Large Language Model Agent: A Survey on Methodology, Applications and Challenges [88.3032929492409]
  Large Language Model (LLM) agents, with goal-driven behaviors and dynamic adaptation capabilities, potentially represent a critical pathway toward artificial general intelligence.<n>This survey systematically deconstructs LLM agent systems through a methodology-centered taxonomy.<n>Our work provides a unified architectural perspective, examining how agents are constructed, how they collaborate, and how they evolve over time.
  arXiv  Detail & Related papers  (2025-03-27T12:50:17Z)
• Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning [50.47568731994238]
  Key method for creating Artificial Intelligence (AI) agents is Reinforcement Learning (RL) This paper presents a general framework model for integrating and learning structured reasoning into AI agents' policies.
  arXiv  Detail & Related papers  (2023-12-22T17:57:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.