A Dataset of Agentic AI Coding Tool Configurations

• URL: http://arxiv.org/abs/2605.08435v1
• Date: Fri, 08 May 2026 19:58:27 GMT
• Title: A Dataset of Agentic AI Coding Tool Configurations
• Abstract summary: dataset covers 4,738 repositories across five tools (Claude Code, GitHub Copilot, OpenAI Codex, Cursor, Gemini) and eight configuration mechanisms.<n>We collected 15,591 configuration artifacts, the full content of 18,167 configuration files associated with these configuration artifacts, and 148,519 AI-co-authored commits.<n>This data supports research on context engineering, AI tool adoption patterns, and human-AI collaboration.
• Score: 11.087963504113937
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Agentic AI coding tools such as Claude Code and OpenAI Codex execute multi-step coding tasks with limited human oversight. To steer these tools, developers create repository-level configuration artifacts (e.g., Markdown files) for configuration mechanisms such as Context Files, Skills, Rules, and Hooks. There is no curated dataset yet that captures these configurations at scale. This dataset, collected from open-source GitHub repositories, fills that gap. We selected 40,585 actively maintained repositories through metadata filtering, classified them using GPT-5.2 to identify 36,710 as belonging to engineered software projects, and systematically detected configuration artifacts in these repositories. The dataset covers 4,738 repositories across five tools (Claude Code, GitHub Copilot, OpenAI Codex, Cursor, Gemini) and eight configuration mechanisms. We collected 15,591 configuration artifacts, the full content of 18,167 configuration files associated with these configuration artifacts, and 148,519 AI-co-authored commits. The dataset and the construction pipeline are publicly available on Zenodo under CC BY 4.0. An interactive website allows researchers to browse and explore the data. This data supports research on context engineering, AI tool adoption patterns, and human-AI collaboration.

Related papers

• OpenAI4S: Code as Action, Science as Sessions [57.428071188574]
  We present OpenAI4S, an open-source scientific research agent built around the principle of emphCode as Action, Science as Sessions.<n>OpenAI4S combines a persistent computing runtime with research-session management orchestration is handled through structured tool calls.<n>We evaluate OpenAI4S on 36 research scenarios spanning retrosynthesis, molecular protein binder design, protein mutation, catalyst screening, and mineral spectroscopy.
  arXiv  Detail & Related papers  (2026-09-14T06:16:59Z)
• Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills [86.15447717549154]
  We present DisCo, a skill-powered research agent that creates skills and uses them during research.<n>We call this missing layer operational knowledge, the know-how that separates knowing a method from making it work.
  arXiv  Detail & Related papers  (2026-09-02T15:49:41Z)
• SpecMine: A Large-Scale Corpus of Spec-Driven Development Artifacts [7.933620345044633]
  SpecMine captures Spec-Driven Development (SDD) in public GitHub repositories.<n>It lets the community study, for the first time, how software is specified in the age of AI agents.
  arXiv  Detail & Related papers  (2026-08-25T22:43:30Z)
• An Exploratory Study of Agent Plans for Agentic AI Coding Tools in Open-Source Software [16.01774662182876]
  Repository-level configuration artifacts allow developers to provide guidance for agentic AI coding tools, such as Claude Code, Gemini, etc.<n>We present an exploratory study of Agent Plans in open-source software repositories, examining how plan files are preserved, which development activities they support, and what information they provide to guide agent execution.
  arXiv  Detail & Related papers  (2026-08-05T10:15:28Z)
• LoopsBench: From Harness Engineering to Loop Engineering in Benchmarking Coding Agent [60.27239351179265]
  LOOPSBENCH is a benchmark for loop engineering in coding agent evaluation.<n>It comprises 112 tasks from authentic sources spanning 8 programming languages and 9 domains.<n>Its flow-aware runtime releases tests along the ready frontier and retains completed nodes as regression obligations.
  arXiv  Detail & Related papers  (2026-07-31T20:18:25Z)
• Context Rot in AI-Assisted Software Development: Repurposing Documentation Consistency for AI Configuration Artifacts [12.261894106066082]
  We argue that developers increasingly provide AI coding assistants with persistent context through configuration files such as CLAUDE.md, AGENTS.md, and.cursorrules.<n>These files describe code elements, architecture, and development conventions, forming the context that guides AI tool behavior across sessions.<n>As software evolves, this context can become stale, a phenomenon we call context rot. While AI configuration artifacts are new, the underlying consistency problem connects to decades of software documentation research.
  arXiv  Detail & Related papers  (2026-06-08T06:36:38Z)
• Configuring Agentic AI Coding Tools: An Exploratory Study [11.643977424519]
  We present a systematic analysis of configuration mechanisms for agentic AI coding tools, covering Claude Code, GitHub Copilot, Cursor, Gemini, and Codex.<n>We identify eight configuration mechanisms and, in an empirical study of 2,926 GitHub repositories, examine whether and how they are adopted.<n>We then explore Context Files, Skills, and Subagents, that is, three mechanisms available across tools, in more detail.
  arXiv  Detail & Related papers  (2026-02-16T12:24:28Z)
• AIDev: Studying AI Coding Agents on GitHub [9.1677959533705]
  We introduce AIDev, a large-scale dataset focused on agent-authored pull requests (Agentic-PRs) in real-world GitHub repositories.<n> AIDev aggregates 932,791 Agentic-PRs produced by five agents: OpenAI Codex, Devin, GitHub Copilot, Cursor, and Claude Code.<n>In addition, AIDev includes a curated subset of 33,596 Agentic-PRs from 2,807 repositories with over 100 stars, providing further information such as comments, reviews, commits, and related issues.
  arXiv  Detail & Related papers  (2026-02-09T20:45:58Z)
• Context Engineering for AI Agents in Open-Source Software [13.236926479239754]
  GenAI-based coding assistants have disrupted software development.<n>Their next generation is agent-based, operating with more autonomy and potentially without human oversight.<n>One challenge is to provide AI agents with sufficient context about the software projects they operate in.
  arXiv  Detail & Related papers  (2025-10-24T12:55:48Z)
• Trace: Securing Smart Contract Repository Against Access Control Vulnerability [58.02691083789239]
  GitHub hosts numerous smart contract repositories containing source code, documentation, and configuration files.<n>Third-party developers often reference, reuse, or fork code from these repositories during custom development.<n>Existing tools for detecting smart contract vulnerabilities are limited in their ability to handle complex repositories.
  arXiv  Detail & Related papers  (2025-10-22T05:18:28Z)
• SwingArena: Competitive Programming Arena for Long-context GitHub Issue Solving [90.32201622392137]
  We present SwingArena, a competitive evaluation framework for Large Language Models (LLMs)<n>Unlike traditional static benchmarks, SwingArena models the collaborative process of software by pairing LLMs as iterations, who generate patches, and reviewers, who create test cases and verify the patches through continuous integration (CI) pipelines.
  arXiv  Detail & Related papers  (2025-05-29T18:28:02Z)
• BigDocs: An Open Dataset for Training Multimodal Models on Document and Code Tasks [57.589795399265945]
  We introduce BigDocs-7.5M, a high-quality, open-access dataset comprising 7.5 million multimodal documents across 30 tasks.<n>We also introduce BigDocs-Bench, a benchmark suite with 10 novel tasks.<n>Our experiments show that training with BigDocs-Bench improves average performance up to 25.8% over closed-source GPT-4o.
  arXiv  Detail & Related papers  (2024-12-05T21:41:20Z)
• Boidae: Your Personal Mining Platform [0.21485350418225244]
  We present Boidae, a family of Boa installations controlled and customized by users. In particular, Boidae allows the creation of custom datasets generated from any set of Git repositories. Boidae's scripts and all infrastructure it builds upon are open-sourced.
  arXiv  Detail & Related papers  (2024-01-20T02:44:47Z)
• Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub [79.31134731122462]
  We introduce OpenAct benchmark to evaluate the open-domain task-solving capability, built on human expert consultation and repositories in GitHub.<n>We present OpenAgent, a novel LLM-based agent system that can tackle evolving queries in open domains through autonomously integrating specialized tools from GitHub.
  arXiv  Detail & Related papers  (2023-12-28T15:47:30Z)
• DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population [95.0099875111663]
  DeepKE implements various information extraction tasks, including named entity recognition, relation extraction and attribute extraction. DeepKE allows developers and researchers to customize datasets and models to extract information from unstructured data according to their requirements.
  arXiv  Detail & Related papers  (2022-01-10T13:29:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.