SkillAlchemy: Open-World Agent Skill Creation

• URL: http://arxiv.org/abs/2608.23417v1
• Date: Mon, 24 Aug 2026 15:58:57 GMT
• Title: SkillAlchemy: Open-World Agent Skill Creation
• Abstract summary: Agent skills are reusable procedural artifacts that extend language agents with specialized, tool conventions, and domain behaviors at inference time.<n>Create reliable skills still depends largely on human authorship, model priors, or execution traces.<n>We propose SkillAlchemy, an admission-centered framework for source-grounded skill creation.
• Score: 11.836010370630568
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agent skills are reusable procedural artifacts that extend language agents with specialized workflows, tool conventions, and domain behaviors at inference time. However, creating reliable skills still depends largely on human authorship, model priors, or execution traces. These sources are often unavailable for unfamiliar tasks, suggesting the need to create skills from open-world materials. In this paper, we study open-world skill creation: given an underspecified skill brief and a source-access specification, a creator must discover behavior-relevant requirements omitted by the brief and determine how broadly each source-derived procedure is justified. We propose SkillAlchemy, an admission-centered framework for source-grounded skill creation. SkillAlchemy identifies implicit requirements through contrastive evidence, admits candidate procedures based on evidence-supported scope, and compiles the admitted content into a grammar-guided skill package. Extensive experiments across 87 SkillsBench v1.1 tasks demonstrate that SkillAlchemy improves pass rate over no-skill execution by 19.9 percentage points and the strongest automated baseline by 8.6 percentage points, while achieving performance comparable to human-curated skills.

Related papers

• SkillShapley: Boundary-Adaptive Shapley Valuation for Skill Step Attribution in LLM Agents [7.7046487215582475]
  Existing agent skills are primarily created through human manual crafting or agent execution traces.<n>We propose SkillShapley, a step-level attribution framework for agent skills.<n> Experiments on skills from the widely adopted SkillsBench demonstrate that SkillShapley can effectively and efficiently identify high- or low-value skill steps.
  arXiv  Detail & Related papers  (2026-08-13T12:41:03Z)
• Generative Skill Composition for LLM Agents [58.47191648555256]
  LLM agents benefit from skills for solving complex tasks.<n>As skill libraries grow, selecting an appropriate skill composition has emerged as a central bottleneck.<n>We propose SkillComposer, which instantiates structured skill composition as task-conditioned skill sequence prediction.
  arXiv  Detail & Related papers  (2026-06-30T17:53:09Z)
• Skill Coverage: A Test Adequacy Metric for Agent Skills [17.60316723069748]
  Agent skills encode reusable procedural knowledge that guides large language model agents across tasks and execution contexts.<n>We introduce skill coverage, a test adequacy metric that treats the skill artifact as the object under test.<n>Applying skill coverage to SkillsBench reveals that existing benchmark executions cover only 39.90 to 43.98% of skill behavior constraints.
  arXiv  Detail & Related papers  (2026-06-09T10:16:05Z)
• SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization [99.18305770138677]
  Agent skills are reusable strategies that guide agent reasoning and action.<n>Current skill construction methods treat the problem as one-shot extraction.<n>We introduce SkillComposer, a framework that decomposes skill construction into three learnable operations.
  arXiv  Detail & Related papers  (2026-06-04T12:16:08Z)
• OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents [15.598856888948093]
  We present textscOpenSkillEval, an automatic evaluation framework for both skill-augmented agent systems and the skills themselves.<n>Instead of relying on static benchmarks, textscOpenSkillEval automatically constructs realistic task instances from evolving real-world artifacts.<n>Using more than 600 dynamically generated task instances and 30 open-source skills, we conduct a systematic evaluation of state-of-the-art models and agent frameworks.
  arXiv  Detail & Related papers  (2026-05-22T14:09:41Z)
• SkillGen: Verified Inference-Time Agent Skill Synthesis [60.927977774369516]
  SkillGen is a framework that synthesizes a single auditable skill from trajectories generated by a base agent.<n>It identifies reusable success patterns, recurring failure modes, and behaviors that appear in nearby successes but are missing from failures.<n>A key novelty in SkillGen is that we model agent skills as interventions to empirically verify the net effect of skills on the overall performance.
  arXiv  Detail & Related papers  (2026-05-09T19:24:11Z)
• From Context to Skills: Can Language Models Learn from Context Skillfully? [56.5857437218136]
  Ctx2Skill is a self-evolving framework that autonomously discovers, refines, and selects context-specific skills.<n>It can be plugged into any language model to obtain better context learning capability.
  arXiv  Detail & Related papers  (2026-04-30T09:53:15Z)
• SkillForge: Forging Domain-Specific, Self-Evolving Agent Skills in Cloud Technical Support [3.277125886726497]
  Existing skill creators lack domain grounding, producing skills poorly aligned with real-world task requirements.<n>We introduce SkillForge, a self-evolving framework that closes the end-to-end creation-refinement loop.<n>We evaluate five real-world cloud support scenarios spanning 1,883 tickets and 3,737 tasks.
  arXiv  Detail & Related papers  (2026-04-09T07:11:23Z)
• EvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification [85.3692584167951]
  Anthropic proposes the concept of skills for LLM agents to tackle multi-step professional tasks.<n>A tool is a single, self-contained function, whereas a skill is a structured bundle of interdependent multi-file artifacts.<n>EvoSkills is a self-evolving skills framework that enables agents to autonomously construct complex, multi-file skill packages.
  arXiv  Detail & Related papers  (2026-04-02T06:43:20Z)
• SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks [61.89812116484928]
  Agent Skills are structured packages of procedural knowledge that augment LLM agents at inference time.<n>We present SkillsBench, a benchmark of 86 tasks across 11 domains paired with curated Skills and deterministic verifiers.<n>We show curated Skills raise average pass rate by 16.2 percentage points(pp), but effects vary widely by domain.<n>Self-generated Skills provide no benefit on average, showing that models cannot reliably author the procedural knowledge they benefit from consuming.
  arXiv  Detail & Related papers  (2026-02-13T07:06:06Z)
• SkillMatch: Evaluating Self-supervised Learning of Skill Relatedness [11.083396379885478]
  We release SkillMatch, a benchmark for the task of skill relatedness based on expert knowledge mining from millions of job ads. We also propose a scalable self-supervised learning technique to adapt a Sentence-BERT model based on skill co-occurrence in job ads.
  arXiv  Detail & Related papers  (2024-10-07T13:05:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.