Skill Drift Is Contract Violation: Proactive Maintenance for LLM Agent Skill Libraries

• URL: http://arxiv.org/abs/2605.10990v1
• Date: Sat, 09 May 2026 11:41:53 GMT
• Title: Skill Drift Is Contract Violation: Proactive Maintenance for LLM Agent Skill Libraries
• Abstract summary: LLM agents increasingly rely on reusable skill libraries, but these skills silently decay as the external services, packages, APIs, and they reference evolve.<n>We formulate skill drift as contract violation and introduce sgname, which extracts executable environment contracts from skill documents.<n>sgname raises zero false alarms over 599 no-drift and hard-negative cases.
• Score: 14.848157882117613
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLM agents increasingly rely on reusable skill libraries, but these skills silently decay as the external services, packages, APIs, and configurations they reference evolve. Existing monitors detect such changes at the wrong granularity: they observe values, not the role those values play in a skill. A version string in a comment is noise; the same string in a pinned dependency is an operational obligation. We formulate skill drift as contract violation and introduce \sgname{}, which extracts executable environment contracts from skill documents and validates only those role-bearing assumptions against known or live conditions. This distinction turns noisy monitoring into a precision-first maintenance signal. Contract-free CI probes produce 40\% false positives, while \sgname{} raises zero false alarms over 599 no-drift and hard-negative cases (Wilson 95\% CI $[0,0.6]\%$). In known-drift verification, \sgname{} achieves 100\% precision and 76\% recall with the strongest backbone; in a pre-registered study over 49 real skills, it discovers live drift with 86\% conservative precision. Violated contracts also make repair actionable, improving one-round success from 10\% without localization to 78\%. We release \dbname{}, an 880-pair benchmark for skill degradation.

Related papers

• When Not to Imitate: Boundary-Aware Skill Memory for Reliable Tool-Use LLM Agents [62.10623192614306]
  Extracting skills from past successes is critical for the efficient evolution of Large Language Model (LLM) agents.<n>We propose textbfBoundary-Aware Skill Memory (BASM), which augments each skill with explicit boundary fields.<n>BASM consistently outperforms success-distilled skill-memory baselines.
  arXiv  Detail & Related papers  (2026-08-23T10:16:06Z)
• Repo2Skill-Evo: Repository Skills Go Stale in Silence [35.22867798999821]
  Large language model (LLM) agents increasingly operate over evolving software repositories.<n>Prior work shows that they can improve agent performance.<n>We study whether agents can keep this externalized knowledge current.
  arXiv  Detail & Related papers  (2026-08-22T14:07:51Z)
• Evaluating Skills, Not Just Agents: Agentic Continuous Evaluation of Skills [9.077159743623076]
  repository-native framework for evaluating skills and product capability packages as executable agent artifacts.<n>ACES runs paired live trials with and without a target skill, grades six default runtime metrics, and reports Skill Lift.<n>Open-source implementation of the methodology is available in NVIDIA SkillEvaluator.
  arXiv  Detail & Related papers  (2026-08-20T23:26:14Z)
• TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation [13.298401993417611]
  TRUSS is an evidence guided framework for generating functionally effective and safety reliable Agent Skills.<n>We evaluate TRUSS on 168 SkillInject artifacts, 155 SkillSafetyBench cases, and all 187 tasks in SkillGenBench.<n>For Skill generation, TRUSS raises task effectiveness from 17.11% without Skills to 52.94%, while increasing the benchmark Security rate from 50.80% to 100.00%.
  arXiv  Detail & Related papers  (2026-08-18T09:52:57Z)
• Demystifying Agent Skills: Why They Work-Until They Don't [58.92875630998418]
  Existing evaluations largely measure whether skills improve aggregated task success, leaving a more fundamental question underexplored: emphtextbfWhen do skills help, why do they work, and where do they fail?<n>We isolate the effects of representation, outcome annotation, retrieval difficulty, and cross-framework of skills.<n>We consolidate these observations into a taxonomy of three high-level categories and twelve skill-use modes.
  arXiv  Detail & Related papers  (2026-08-14T07:26:38Z)
• SLBench: Evaluating How LLM Agents Follow Logical Relations in Skills [15.716453891551057]
  We introduce SkillLogic, a framework for analyzing logical relations in skill files.<n>Using SkillLogic, we scan over 5000 public skills and find that 70% contain at least one logical relation.<n>We then construct SLBench, an 86-case executable benchmark from high-confidence, high-impact, and locally testable relations.
  arXiv  Detail & Related papers  (2026-07-10T00:48:30Z)
• SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing [81.51044612408793]
  SkillAudit is a framework for evolving agent skills without ground-truth feedback.<n>To turn behavioral differences into edit guidance, SkillAudit uses Process-Aligned Contrastive Evaluation.<n>It routes edits through two pipelines: Refine removes noisy or irrelevant guidance from broadly useful skills, while Repair replaces passages that conflict with the task.
  arXiv  Detail & Related papers  (2026-06-12T08:20:09Z)
• EvoPoC: Automated Exploit Synthesis for DeFi Smart Contracts via Hierarchical Knowledge Graphs [11.754368842084412]
  sys is a knowledge-driven agentic system for end-to-end contract vulnerability detection and exploit synthesis.<n>sys was evaluated on 88 real-world DeFi attacks and 72 audited projects (2,573 contracts)<n>sys identified 16 confirmed 0-day vulnerabilities, helping secure over $70.6M and earning $2,900 in bounties.
  arXiv  Detail & Related papers  (2026-05-04T17:43:52Z)
• Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows [67.92316850084575]
  We introduce Claw-Eval-Live, a live benchmark for workflow agents.<n>Each release is constructed from public workflow-demand signals.<n>Claw-Eval-Live records execution traces, audit logs, service state, and post-run workspace artifacts.
  arXiv  Detail & Related papers  (2026-04-30T17:23:19Z)
• Credential Leakage in LLM Agent Skills: A Large-Scale Empirical Study [51.717224133855886]
  Third-party skills extend LLM agents with powerful capabilities but often handle sensitive credentials in privileged environments.<n>We analyze 17,022 skills (sampled from 170,226 on SkillsMP) using static analysis, sandbox testing, and manual inspection.<n>We identify 520 vulnerable skills with 1,708 issues and derive a taxonomy of 10 leakage patterns (4 accidental and 6 adversarial)
  arXiv  Detail & Related papers  (2026-04-03T14:50:16Z)
• ContractSkill: Repairable Contract-Based Skills for Multimodal Web Agents [13.2889725132666]
  We present ContractSkill, a framework that converts a draft skill into a contracted executable artifact.<n>This representation enables deterministic verification, step-level fault localization, and minimal patch-based repair.
  arXiv  Detail & Related papers  (2026-03-20T09:25:50Z)
• SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agents [52.20768003832476]
  We analyze execution traces on $$-Bench (Airline/Retail) and SWE-Bench Verified.<n>We formalize emphdecisive deviations, earliest action, level divergences that flip success to failure.<n>We introduce cm, a model-agnostic, gradient-free, test-time safeguard.
  arXiv  Detail & Related papers  (2025-11-26T01:28:22Z)
• One Signature, Multiple Payments: Demystifying and Detecting Signature Replay Vulnerabilities in Smart Contracts [56.94148977064169]
  lacking checks on signature usage conditions can lead to repeated verifications, increasing the risk of permission abuse and threatening contract assets.<n>We define this issue as the Signature Replay Vulnerability (SRV)<n>From 1,419 audit reports across 37 blockchain security companies, we identified 108 with detailed SRV descriptions and classified five types of SRVs.
  arXiv  Detail & Related papers  (2025-11-12T09:17:13Z)
• Trace: Securing Smart Contract Repository Against Access Control Vulnerability [58.02691083789239]
  GitHub hosts numerous smart contract repositories containing source code, documentation, and configuration files.<n>Third-party developers often reference, reuse, or fork code from these repositories during custom development.<n>Existing tools for detecting smart contract vulnerabilities are limited in their ability to handle complex repositories.
  arXiv  Detail & Related papers  (2025-10-22T05:18:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.