From Prompt to Process: a Process Taxonomy and Comparative Assessment of Frameworks Supporting AI Software Development Agents

• URL: http://arxiv.org/abs/2606.04967v1
• Date: Wed, 03 Jun 2026 14:49:15 GMT
• Title: From Prompt to Process: a Process Taxonomy and Comparative Assessment of Frameworks Supporting AI Software Development Agents
• Abstract summary: We focus on the operational frameworks that turn AI capabilities into process.<n>Each attack AI development through a different path: spec-driven development in full and lightweight variants, agent-driven agile planning, context engineering over the agent, worktree isolation and review, and recovery of operational specifications from legacy systems.<n>Our central contribution is a six-dimension process taxonomy: specification, context, roles, execution, validation and portability, with a scoring rubric that turns it into a replicable instrument.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI tools for programming are no longer just autocomplete or chat assistants: they organize themselves as development frameworks, with process, roles, artifacts and verification. Recent surveys map agents and LLMs for software engineering, but a study centered on the operational frameworks that turn these capabilities into process is missing. We ran a directed search of primary sources, with a functional inclusion criterion and traction measurement, and selected six frameworks: GitHub Spec Kit, OpenSpec, BMAD Method, Get Shit Done (GSD), Spec Kitty and Reversa. Each attacks AI development through a different path: spec-driven development in full and lightweight variants, agent-driven agile planning, context engineering over the agent, worktree isolation and review, and recovery of operational specifications from legacy systems. Our central contribution is a six-dimension process taxonomy: specification, context, roles, execution, validation and portability, with a scoring rubric that turns it into a replicable instrument. We apply it to the six frameworks and an out-of-sample case, Spec-Flow. Two results stand out. Among frameworks that already adopt some process there is convergence: the isolated prompt loses centrality, and persistent artifacts, work contracts, traceability and human review become mechanisms that reduce ambiguity and coordinate agents. And no framework strongly covers all six dimensions, exposing a structural trade-off between process depth and portability across agents. We also found recurring risks: drift between specification and code, excessive trust in generated artifacts, fragility of community extensions, platform dependence and a lack of benchmarks for the complete process. We close with a research agenda for empirical evaluation, focused on intermediate-quality metrics, context governance, installation security and reproducibility.

Related papers

• Self-Evolving Coding Agents [11.43197414613246]
  Self-evolving coding agents can inspect repositories, invoke tools, execute tests, debug failures, and generate patches.<n>We first define self-evolving coding agents and distinguish them from conventional coding agents and general self-evolving agents.<n>We find that executable feedback, repository-level context, and coding trajectories give software engineering a distinctive role as a natural domain for agent self-evolution.
  arXiv  Detail & Related papers  (2026-08-04T09:43:46Z)
• Using Process Mining to Generate AI Agents from Software Engineering Process Records [1.6974371408199849]
  We address the challenge of integrating AI agents into Software Engineering (SE) teams.<n>Our approach discovers project-specific agent roles using a predefined SE role vocabulary.<n>As proof-of-concept, we applied our approach to a well-established open-source project.
  arXiv  Detail & Related papers  (2026-07-06T11:22:25Z)
• From Question Answering to Task Completion: A Survey on Agent System and Harness Design [67.8241530947325]
  This survey examines LLM-based agents through a model-harness lens.<n>We first clarify the functional definition of agents and the implementation view of an LLM-based agent as a foundation model and an execution harness.<n>We then analyze the limits of model-centric scaling, trace four paradigms of agent engineering, and decompose the execution harness into six coupled runtime responsibilities.
  arXiv  Detail & Related papers  (2026-06-14T05:40:16Z)
• Code as Agent Harness [107.31925305395957]
  In emerging agentic systems, code is no longer only a target output.<n>Code increasingly serves as an operational substrate for agent reasoning, acting, environment modeling, and execution-based verification.<n>This survey provides a unified roadmap toward executable, verifiable, and stateful AI agent systems.
  arXiv  Detail & Related papers  (2026-05-18T17:59:03Z)
• AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents [1.4323566945483497]
  Foundation models have transformed automated code generation, yet autonomous software-engineering agents remain unreliable in realistic development settings.<n>We propose a model-harness-environment system in which a runtime substrate mediates how a foundation-model agent observes a project, acts on it, receives feedback, and establishes that a change is complete.<n>The framework reframes the central question of autonomous software engineering from whether a foundation model can produce a patch to whether the model-harness-environment system can produce a verifiably correct, attributed, and maintainable change.
  arXiv  Detail & Related papers  (2026-05-13T11:14:59Z)
• ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development [72.4729759618632]
  We introduce ABC-Bench, a benchmark to evaluate agentic backend coding within a realistic, executable workflow.<n>We curated 224 practical tasks spanning 8 languages and 19 frameworks from open-source repositories.<n>Our evaluation reveals that even state-of-the-art models struggle to deliver reliable performance on these holistic tasks.
  arXiv  Detail & Related papers  (2026-01-16T08:23:52Z)
• AI Agent Systems: Architectures, Applications, and Evaluation [4.967019713320407]
  AI agents combine foundation models with reasoning, planning, memory, and tool use.<n>We organize prior work into a unified taxonomy spanning agent components.<n>We discuss key design trade-offs -- latency vs. accuracy, autonomy vs. controllability, and capability vs. reliability.
  arXiv  Detail & Related papers  (2026-01-05T02:38:40Z)
• Multi-Agent Systems for Dataset Adaptation in Software Engineering: Capabilities, Limitations, and Future Directions [8.97512410819274]
  This paper presents the first empirical study on how state-of-the-art multi-agent systems perform in dataset adaptation tasks.<n>We evaluate GitHub Copilot on adapting SE research artifacts from benchmark repositories including ROCODE and LogHub2.0.<n>Results show that current systems can identify key files and generate partial adaptations but rarely produce correct implementations.
  arXiv  Detail & Related papers  (2025-11-26T13:26:11Z)
• LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering [90.84806758077536]
  We introduce textbfLoCoBench-Agent, a comprehensive evaluation framework specifically designed to assess large language models (LLMs) agents in realistic, long-context software engineering.<n>Our framework extends LoCoBench's 8,000 scenarios into interactive agent environments, enabling systematic evaluation of multi-turn conversations.<n>Our framework provides agents with 8 specialized tools (file operations, search, code analysis) and evaluates them across context lengths ranging from 10K to 1M tokens.
  arXiv  Detail & Related papers  (2025-11-17T23:57:24Z)
• A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System [56.40989626804489]
  This survey provides the first holistic analysis of Large Language Models-powered software engineering.<n>We review over 150 recent papers and propose a taxonomy along two key dimensions: (1) Solutions, categorized into prompt-based, fine-tuning-based, and agent-based paradigms, and (2) Benchmarks, including tasks such as code generation, translation, and repair.
  arXiv  Detail & Related papers  (2025-10-10T06:56:50Z)
• AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities [8.086360127362815]
  Large language model (LLM)-based coding agents autonomously plan, execute, and interact with tools such as compilers, debuggers, and version control systems.<n>Unlike conventional code generation, these agents decompose goals, coordinate multi-step processes, and adapt based on feedback, reshaping software development practices.
  arXiv  Detail & Related papers  (2025-08-15T00:14:31Z)
• Deep Research Agents: A Systematic Examination And Roadmap [109.53237992384872]
  Deep Research (DR) agents are designed to tackle complex, multi-turn informational research tasks.<n>In this paper, we conduct a detailed analysis of the foundational technologies and architectural components that constitute DR agents.
  arXiv  Detail & Related papers  (2025-06-22T16:52:48Z)
• HADA: Human-AI Agent Decision Alignment Architecture [0.0]
  HADA is a protocol- and framework reference architecture that keeps both large language model (LLM) agents and legacy algorithms aligned with organizational targets and values.<n>Technical and non-technical actors can query, steer, audit, or contest every decision across strategic, tactical, and real-time horizons.
  arXiv  Detail & Related papers  (2025-06-01T14:04:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.