The Impact of Configuring Agentic AI Coding Tools on Build-vs-Buy Decisions: A Study Protocol

• URL: http://arxiv.org/abs/2606.03907v1
• Date: Tue, 02 Jun 2026 17:01:28 GMT
• Title: The Impact of Configuring Agentic AI Coding Tools on Build-vs-Buy Decisions: A Study Protocol
• Abstract summary: Agentic AI coding tools write code with increasing autonomy and decide when to import a library and when to implement functionality from scratch.<n>No controlled experimental study has examined what governs build-versus-buy decisions in agentic AI coding tools.<n>We present a pre-registered protocol to study how configuration mechanisms alter build-versus-buy behavior in two popular agentic AI coding tools: Claude Code and OpenAI Codex.
• Score: 17.730557321084408
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Agentic AI coding tools write code with increasing autonomy and in doing so decide when to import a library and when to implement functionality from scratch. These decisions, whether to build functionality from scratch or buy into an external library, hereafter build-versus-buy, carry direct consequences for software security, licensing compliance, performance, and long-term maintainability. Yet no controlled experimental study has examined what governs build-versus-buy decisions in agentic AI coding tools. Configuration mechanisms, i.e., the means by which developers tailor agentic AI coding tool behavior to a project or workflow, are one of the primary means by which practitioners can influence these decisions. However, it is unclear which configuration mechanisms influence build-versus-buy decisions most effectively. We present a pre-registered protocol to study how configuration mechanisms alter build-versus-buy behavior in two popular agentic AI coding tools: Claude Code and OpenAI Codex. We will execute controlled programming tasks drawn from a benchmark of staged projects, each constructed around identifiable build-versus-buy points, and will manipulate the configuration supplied to each tool, ranging from no configuration, through context files with soft preferences and explicit prohibitions, to Skills (instructions that can be autonomously discovered), MCP-enabled library discovery tools, and permission controls, measuring which libraries the tool selects, whether it discloses newly introduced libraries, and whether those disclosures are complete and accurate. Nine pre-registered hypotheses structure the protocol. The resulting benchmark dataset and analysis pipeline will be released as a reusable artifact for evaluating build-versus-buy behavior in agentic AI coding tools.

Related papers

• Using Process Mining to Generate AI Agents from Software Engineering Process Records [1.6974371408199849]
  We address the challenge of integrating AI agents into Software Engineering (SE) teams.<n>Our approach discovers project-specific agent roles using a predefined SE role vocabulary.<n>As proof-of-concept, we applied our approach to a well-established open-source project.
  arXiv  Detail & Related papers  (2026-07-06T11:22:25Z)
• ToolRosetta: Bridging Open-Source Repositories and Large Language Model Agents through Automated Tool Standardization [51.92237664440418]
  ToolRosetta is a framework that automatically translates open-source code repositories and APIs into MCP-compatible tools.<n>Given a user task, ToolRosetta autonomously plans toolchains, identifies relevant toolchains, and converts them into executable MCP services.
  arXiv  Detail & Related papers  (2026-03-10T07:19:43Z)
• OpenSage: Self-programming Agent Generation Engine [56.399761469404496]
  We propose OpenSage, the first agent development kit (ADK) to automatically create agents with self-generated topology and toolsets.<n>OpenSage offers effective functionality for agents to create and manage their own sub-agents and toolkits.<n>We believe OpenSage can pave the way for the next generation of agent development, shifting the focus from human-centered to AI-centered paradigms.
  arXiv  Detail & Related papers  (2026-02-18T21:16:29Z)
• Configuring Agentic AI Coding Tools: An Exploratory Study [11.643977424519]
  We present a systematic analysis of configuration mechanisms for agentic AI coding tools, covering Claude Code, GitHub Copilot, Cursor, Gemini, and Codex.<n>We identify eight configuration mechanisms and, in an empirical study of 2,926 GitHub repositories, examine whether and how they are adopted.<n>We then explore Context Files, Skills, and Subagents, that is, three mechanisms available across tools, in more detail.
  arXiv  Detail & Related papers  (2026-02-16T12:24:28Z)
• Confucius Code Agent: Scalable Agent Scaffolding for Real-World Codebases [44.366968508477235]
  We introduce the Confucius Code Agent (CCA), a scalable software engineering agent that can operate at large-scales.<n>CCA is built on top of the Confucius SDK, an agent development platform structured around three complementary perspectives.<n>In addition, we introduce a meta-agent that automates the synthesis, evaluation, and refinement of agent configurations.
  arXiv  Detail & Related papers  (2025-12-11T08:05:58Z)
• DeepAgent: A General Reasoning Agent with Scalable Toolsets [111.6384541877723]
  DeepAgent is an end-to-end deep reasoning agent that performs autonomous thinking, tool discovery, and action execution.<n>To address the challenges of long-horizon interactions, we introduce an autonomous memory folding mechanism that compresses past interactions into structured episodic, working, and tool memories.<n>We develop an end-to-end reinforcement learning strategy, namely ToolPO, that leverages LLM-simulated APIs and applies tool-call advantage attribution to assign fine-grained credit to the tool invocation tokens.
  arXiv  Detail & Related papers  (2025-10-24T16:24:01Z)
• Context Engineering for AI Agents in Open-Source Software [13.236926479239754]
  GenAI-based coding assistants have disrupted software development.<n>Their next generation is agent-based, operating with more autonomy and potentially without human oversight.<n>One challenge is to provide AI agents with sufficient context about the software projects they operate in.
  arXiv  Detail & Related papers  (2025-10-24T12:55:48Z)
• ToolLibGen: Scalable Automatic Tool Creation and Aggregation for LLM Reasoning [80.10274552177096]
  Large Language Models (LLMs) equipped with external tools have demonstrated enhanced performance on complex reasoning tasks.<n>The widespread adoption of this tool-augmented reasoning is hindered by the scarcity of domain-specific tools.<n>We propose a systematic approach to automatically an unstructured collection of tools into a structured tool library.
  arXiv  Detail & Related papers  (2025-10-09T04:11:16Z)
• LLM Agents Making Agent Tools [2.5529148902034637]
  Tool use has turned large language models (LLMs) into powerful agents that can perform complex multi-step tasks.<n>But these tools must be implemented in advance by human developers.<n>We propose ToolMaker, an agentic framework that autonomously transforms papers with code into LLM-compatible tools.
  arXiv  Detail & Related papers  (2025-02-17T11:44:11Z)
• ToolCoder: A Systematic Code-Empowered Tool Learning Framework for Large Language Models [81.12673534903979]
  Tool learning has emerged as a crucial capability for large language models (LLMs) to solve complex real-world tasks through interaction with external tools.<n>We propose ToolCoder, a novel framework that reformulates tool learning as a code generation task.
  arXiv  Detail & Related papers  (2025-02-17T03:42:28Z)
• Commit0: Library Generation from Scratch [77.38414688148006]
  Commit0 is a benchmark that challenges AI agents to write libraries from scratch.<n>Agents are provided with a specification document outlining the library's API as well as a suite of interactive unit tests.<n> Commit0 also offers an interactive environment where models receive static analysis and execution feedback on the code they generate.
  arXiv  Detail & Related papers  (2024-12-02T18:11:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.