Who Maintains Agent Skills? A Longitudinal Study of Human-Governed, AI-Assisted Skill Maintenance
- URL: http://arxiv.org/abs/2609.05677v1
- Date: Fri, 04 Sep 2026 19:18:18 GMT
- Title: Who Maintains Agent Skills? A Longitudinal Study of Human-Governed, AI-Assisted Skill Maintenance
- Abstract summary: Recent work seeks to automate skill curation, but it largely evaluates against automated baselines and treats human maintenance as an unmeasured bottleneck.<n>We mine the full commit histories of five public AI-skill repositories, a purposive sample of AI-tooling organizations, covering 873 commits, 143 skill files, and 254 substantive post-creation edits from October 2025 to June 2026.<n>First, every substantive edit is authored or merged through a named human account, while 62% carry an AI co-author trailer, with large repository-level variation.
- Score: 10.648969524722448
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Lifelong LLM agents increasingly rely on external skill artifacts as one element for preserving and reusing capabilities over time. These skills (usually portable Markdown files such as SKILL.md) describe when and how to apply a capability and must be corrected, expanded, and consolidated as tools and usage patterns shift over deployment. Recent work seeks to automate skill curation, but it largely evaluates against automated baselines and treats human maintenance as an unmeasured bottleneck. We study that missing process directly. We mine the full commit histories of five public AI-skill repositories, a purposive sample of AI-tooling organizations, covering 873 commits, 143 skill files, and 254 substantive post-creation edits from October 2025 to June 2026. We code each edit with pre-registered governance, operation, and trigger-evidence codebooks. Three findings emerge. First, every substantive edit is authored or merged through a named human account, while 62% carry an AI co-author trailer, with large repository-level variation. Second, these edits are genuine curation: an audited sample shows that most change skill content, and the coded operations are dominated by additions and corrections. Third, a pre-registered rule-likeness axis fails its reliability gate; reliably coding rule-likeness from commit artifacts remains an open measurement problem. We release the corpus, codebooks, mining scripts, and a replay protocol for automated skill curators. For self-evolving agents, public skill maintenance currently looks less like an autonomous pipeline than a human-governed, AI-assisted loop that future curators must measure against and operate within.
Related papers
- From Registry to Repository: How AI Agent Skills Are Written, Adapted, and Maintained [13.991357454554949]
We present the first empirical study of AI agent skills as engineered artefacts that are authored, reused, customised and maintained.<n>Most reused skills remain near-verbatim, 53% are never modified after adoption, and subsequent local maintenance is overwhelmingly additive.
arXiv Detail & Related papers (2026-07-01T13:14:45Z) - SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing [81.51044612408793]
SkillAudit is a framework for evolving agent skills without ground-truth feedback.<n>To turn behavioral differences into edit guidance, SkillAudit uses Process-Aligned Contrastive Evaluation.<n>It routes edits through two pipelines: Refine removes noisy or irrelevant guidance from broadly useful skills, while Repair replaces passages that conflict with the task.
arXiv Detail & Related papers (2026-06-12T08:20:09Z) - VASO: Formally Verifiable Self-Evolving Skills for Physical AI Agents [57.240036084348354]
We introduce VASO, a framework for verification-guided self-evolution of robot skill contracts.<n>VASO verifies logically inconsistent skill contracts, then verifies plans induced by the skill against global and local temporal specifications.<n>On Clearpath Jackal and PX4 quadcopter tasks, VASO reaches 97.2% formal-specification compliance using fewer than 100 optimization samples.
arXiv Detail & Related papers (2026-06-03T20:02:35Z) - COLLEAGUE.SKILL: Automated AI Skill Generation via Expert Knowledge Distillation [58.84646485020439]
We present an automated trace-to-skill distillation system for generating person-grounded AI skills.<n> COLLEAGUE.SKILL produces a versioned skill package with two tracks: a capability track for practices, mental models, and decisions, and a bounded behavior track for communication style, interaction rules, and correction history.
arXiv Detail & Related papers (2026-05-29T12:59:08Z) - SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphs [70.1970574147839]
We propose SKILLGRAPH, a framework that represents reusable skills as nodes in a directed graph.<n>SKILLGRAPH retrieves not just individual skills, but an ordered skill subgraph that can guide multi-step decision making.<n> Experiments show that SKILLGRAPH achieves state-of-the-art performance against memory-augmented RL methods.
arXiv Detail & Related papers (2026-05-12T12:21:49Z) - To What Extent Does Agent-generated Code Require Maintenance? An Empirical Study [0.0]
This study empirically investigates the maintenance extent, human involvement, and modification types of AI-generated files versus human-authored code.<n>We analyzed over 1,000 files and approximately 3,200 changes from 100 popular repositories.
arXiv Detail & Related papers (2026-05-07T15:52:41Z) - SWE-chat: Coding Agent Interactions From Real Users in the Wild [70.18158706281724]
SWE-chat is the first large-scale dataset of real coding agent sessions collected from open-source developers in the wild.<n>The dataset currently contains 6,000 sessions, comprising more than 63,000 user prompts and 355,000 agent tool calls.
arXiv Detail & Related papers (2026-04-22T17:08:19Z) - Early-Stage Prediction of Review Effort in AI-Generated Pull Requests [0.0]
We analyze 33,707 agent-authored PRs from the AIDev dataset across 2,807 repositories.<n>We propose a Circuit Breaker triage model that predicts high-review-effort PRs at creation time.
arXiv Detail & Related papers (2026-01-02T17:18:01Z) - Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [60.04362496037186]
We present the first controlled study of developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants.<n>Our results show agents can assist developers in ways that surpass copilots.
arXiv Detail & Related papers (2025-07-10T20:12:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.