Do Personalized Skills Help Coding Agents? An Empirical Study of Developer Interaction Histories

• URL: http://arxiv.org/abs/2608.10319v2
• Date: Sat, 15 Aug 2026 06:04:23 GMT
• Title: Do Personalized Skills Help Coding Agents? An Empirical Study of Developer Interaction Histories
• Abstract summary: We propose a framework for extracting reusable developer preferences from interaction traces.<n>It first generates personalized skills through rule-based bootstrapping and evidence-grounded refinement.<n>We compare personalized skills against no-skill, generic-skill, and other-user-skill baselines.
• Score: 5.339166296276375
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM)-powered agents have rapidly evolved from code-completion tools into solvers of complex software engineering tasks. As developers collaborate with coding agents over time, their preferences emerge through repeated interactions and can be used to adapt agent behavior to better meet individual developers' needs. Capturing and reusing these preferences may reduce repeated corrections and improve developer-agent collaboration. Agent skills provide a lightweight mechanism for transferring experience without modifying model parameters. However, existing work primarily focuses on task-specific skills, and it remains unclear whether developer-specific skills distilled from interaction histories can generalize to future tasks. We propose a framework for extracting reusable developer preferences from interaction traces. It first generates personalized skills through rule-based bootstrapping and evidence-grounded refinement, and then evaluates them using a reproducible replay framework with an interactive, trajectory-conditioned LLM-based human developer simulator. We conduct an experiment on 206 real-world developer-agent sessions from 13 developers and compare personalized skills against no-skill, generic-skill, and other-user-skill baselines. Personalized skills provide small and inconsistent improvements over the no-skill baseline, whereas generic skills pooled across developers achieve the largest and most consistent gains. Further analysis suggests that personalized skills become more effective when developer preferences appear frequently, particularly when their histories contain multiple examples relevant to future tasks. These findings provide empirical insights into when developer-specific personalization is effective and demonstrate that broadly transferable procedural knowledge can be more robust than developer-specific preference signals.

Related papers

• Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents [67.97365535829098]
  Learning from experience is critical for developing capable, self-improving large language model (LLM) agents.<n>We propose Evo-Harness, which distills noisy, single-shot executions into reusable skill harnesses for cross-domain and topic-level adaptation.<n>Our analysis demonstrates the effectiveness of Evo-Harness and provides a principled understanding of how LLM agents can effectively learn on the fly.
  arXiv  Detail & Related papers  (2026-08-15T06:43:56Z)
• Self-Evolving Coding Agents [11.43197414613246]
  Self-evolving coding agents can inspect repositories, invoke tools, execute tests, debug failures, and generate patches.<n>We first define self-evolving coding agents and distinguish them from conventional coding agents and general self-evolving agents.<n>We find that executable feedback, repository-level context, and coding trajectories give software engineering a distinctive role as a natural domain for agent self-evolution.
  arXiv  Detail & Related papers  (2026-08-04T09:43:46Z)
• From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents [56.187590188698955]
  Large Language Models have enabled increasingly capable autonomous agents, yet personalization remains critical for making such agents practically useful.<n>Recent benchmarks have begun evaluating personalization in agents, but they largely rely on static preference snapshots, fixed interaction logs, or question answering over predefined user profiles.<n>We introduce IBA-Bench, a benchmark for implicit behavioral alignment constructed from longitudinal interaction histories.<n>We propose IBA-Agent, an agent framework that reconciles conflicting priorities through broad retrieval and trajectory-level alignment.
  arXiv  Detail & Related papers  (2026-08-03T12:52:04Z)
• On the Road to Personalized Code Intelligence: Portraiting and Assisting Developers Based on Their In-IDE Behaviors [38.53670556517792]
  VirtualME is a novel data infrastructure designed to model the developer by continuously capturing and interpreting their dynamic programming behaviors and preferences.<n>On top of VirtualME, we propose a solution for personalized repository-level knowledge Q&A by integrating the developer persona into the Q&A agent.
  arXiv  Detail & Related papers  (2026-05-28T05:17:08Z)
• VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactions [63.13827503828231]
  We introduce VitaBench 2.0, a benchmark for evaluating personalized and proactive agent behavior in long-term user interactions.<n>Results show that real-world personalization remains highly challenging even for state-of-the-art models.
  arXiv  Detail & Related papers  (2026-05-26T15:07:38Z)
• Swarm Skills: A Portable, Self-Evolving Multi-Agent System Specification for Coordination Engineering [50.72456732323037]
  We propose textbfSwarm Skills, a portable specification that extends the Anthropic Skills standard with multi-agent semantics.<n>Swarm Skills turns multi-agent into first-class, distributable assets that consist of roles, execution bounds, and a built-in semantic structure for self-evolution.<n>We show how Swarm Skills achieves zero-adapter cross-agent portability via progressive disclosure, enabling agent teams to self-evolve their coordination strategies without framework lock-in.
  arXiv  Detail & Related papers  (2026-05-11T06:26:58Z)
• SkillClaw: Let Skills Evolve Collectively with Agentic Evolver [18.942639318503062]
  We present SkillClaw, a framework for collective skill evolution in multi-user agent ecosystems.<n> SkillClaw treats cross-user and over-time interactions as the primary signal for improving skills.<n>It continuously aggregates trajectories generated during use and processes them with an autonomous evolver.<n>The resulting skills are maintained in a shared repository and synchronized across users.
  arXiv  Detail & Related papers  (2026-04-09T15:38:27Z)
• Are We All Using Agents the Same Way? An Empirical Study of Core and Peripheral Developers Use of Coding Agents [4.744786007044749]
  We study how core and peripheral developers use, review, modify, and verify agent-generated contributions prior to acceptance.<n>A subset of peripheral developers use agents more often, delegating tasks evenly across bug fixing, feature addition, documentation, and testing.<n>In contrast, core developers focus more on documentation and testing, yet their agentic PRs are frequently merged into the main/master branch.
  arXiv  Detail & Related papers  (2026-01-27T22:50:01Z)
• Enabling Personalized Long-term Interactions in LLM-based Agents through Persistent Memory and User Profiles [0.4885400580268118]
  Large language models (LLMs) increasingly serve as the central control unit of AI agents.<n>We present a framework that integrates persistent memory, dynamic coordination, self-validation, and evolving user profiles to enable personalized long-term interactions.
  arXiv  Detail & Related papers  (2025-10-09T08:22:16Z)
• Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [60.04362496037186]
  We present the first controlled study of developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants.<n>Our results show agents can assist developers in ways that surpass copilots.
  arXiv  Detail & Related papers  (2025-07-10T20:12:54Z)
• Experiential Co-Learning of Software-Developing Agents [83.34027623428096]
  Large language models (LLMs) have brought significant changes to various domains, especially in software development. We introduce Experiential Co-Learning, a novel LLM-agent learning framework. Experiments demonstrate that the framework enables agents to tackle unseen software-developing tasks more effectively.
  arXiv  Detail & Related papers  (2023-12-28T13:50:42Z)
• AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems [112.76941157194544]
  We propose AgentCF for simulating user-item interactions in recommender systems through agent-based collaborative filtering. We creatively consider not only users but also items as agents, and develop a collaborative learning approach that optimize both kinds of agents together. Overall, the optimized agents exhibit diverse interaction behaviors within our framework, including user-item, user-user, item-item, and collective interactions.
  arXiv  Detail & Related papers  (2023-10-13T16:37:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.