RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources

• URL: http://arxiv.org/abs/2606.29538v1
• Date: Sun, 28 Jun 2026 17:59:21 GMT
• Title: RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources
• Abstract summary: 2SKILL is a framework that distills multimodal resources, including tutorial videos, articles, and reference artifacts, into executable skills for software agents.<n>It organizes these skills as a hierarchical multimodal Skill Wiki, where each entry combines structured text, code, visual examples, metadata, and provenance.<n>At time, agents retrieve and compose relevant skills from the wiki; when coverage is insufficient, the same operator can acquire new skills online.
• Score: 34.489558310096776
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Skills are a useful abstraction for software agents, turning human and agent experience into reusable procedural knowledge. Yet existing skill libraries are mostly hand-written, text-centric, or derived from agent traces, leaving tutorial videos and other multimodal human resources largely underused. We present RESOURCE2SKILL, a framework that distills multimodal resources, including tutorial videos, repositories, articles, and reference artifacts, into executable skills for software agents. RESOURCE2SKILL organizes these skills as a hierarchical multimodal Skill Wiki, where each entry combines structured text, code, visual examples, metadata, and provenance. This design preserves complementary signals from different resources: videos capture temporal operations and visual effects, code captures executable tool patterns, and articles or artifacts provide conceptual and stylistic grounding. At inference time, agents retrieve and compose relevant skills from the wiki; when coverage is insufficient, the same construction operator can acquire new skills online. Across seven practical authoring domains, RESOURCE2SKILL improves average overall score by +11.9 percentage points over no-skill agents and outperforms strong harness baselines in 26 of 28 main-aggregate model-domain cells. Ablations confirm the value of multimodal skill format, hierarchical organization, source diversity, selection strategy, and online acquisition.

Related papers

• PRAXIS: Graph-Grounded Tacit Knowledge for Domain Code Generation [61.752289169878566]
  We identify the root cause of the agent's lack of tacit knowledge, including domain-specific business rules, interface contracts, and operational conventions that developers internalize through practice but never document.<n>This knowledge is deeply buried beneath the domain code, across code entities and their dependency relations, and invisible to the agent that lacks it.<n>We propose PRAXIS, a framework that enables agents to extract, represent, and reuse tacit knowledge for domain code generation.
  arXiv  Detail & Related papers  (2026-08-20T08:28:28Z)
• Generative Skill Composition for LLM Agents [58.47191648555256]
  LLM agents benefit from skills for solving complex tasks.<n>As skill libraries grow, selecting an appropriate skill composition has emerged as a central bottleneck.<n>We propose SkillComposer, which instantiates structured skill composition as task-conditioned skill sequence prediction.
  arXiv  Detail & Related papers  (2026-06-30T17:53:09Z)
• Beyond Domains: Reusing Web Skills via Transferable Interaction Patterns [68.0603867264595]
  SkillMigrator learns reusable web skills and transfers them across sites by matching layout structure rather than specific element references.<n>It reduces the average LLM-action count on successful trajectories by 8-10% across both WebArena and Mind2Web at matched success rate.
  arXiv  Detail & Related papers  (2026-06-16T08:04:35Z)
• SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization [99.18305770138677]
  Agent skills are reusable strategies that guide agent reasoning and action.<n>Current skill construction methods treat the problem as one-shot extraction.<n>We introduce SkillComposer, a framework that decomposes skill construction into three learnable operations.
  arXiv  Detail & Related papers  (2026-06-04T12:16:08Z)
• Demo2Tutorial: From Human Experience to Multimodal Software Tutorials [63.678709079907684]
  Demo2Tutorial is a framework that transforms experience captured via screen recordings and interaction logs into structured, multimodal software tutorials.<n>A Step Planner then abstracts these steps into hierarchical task graphs representing goals and steps.<n>A Composer transforms the parsed experience into structured, reusable image-text instructions.
  arXiv  Detail & Related papers  (2026-06-02T17:39:07Z)
• SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphs [70.1970574147839]
  We propose SKILLGRAPH, a framework that represents reusable skills as nodes in a directed graph.<n>SKILLGRAPH retrieves not just individual skills, but an ordered skill subgraph that can guide multi-step decision making.<n> Experiments show that SKILLGRAPH achieves state-of-the-art performance against memory-augmented RL methods.
  arXiv  Detail & Related papers  (2026-05-12T12:21:49Z)
• SkillRAE: Agent Skill-Based Context Compilation for Retrieval-Augmented Execution [10.83969404435554]
  Large Language Model (LLM)-based agents increasingly rely on reusable skill libraries to solve artifact-rich tasks.<n>We propose SkillRAE, a two-stage RAE approach focusing on skill-based context compilation.<n> Experiments on two public benchmarks show that SkillRAE achieves a significant improvement over baselines for RAE.
  arXiv  Detail & Related papers  (2026-05-11T07:31:48Z)
• SKILLFOUNDRY: Building Self-Evolving Agent Skill Libraries from Heterogeneous Scientific Resources [5.7282243774488295]
  We present SkillFoundry, a self-evolving framework that converts resources into validated agent skills.<n>We show that SkillFoundry produces a substantially novel and internally valid skill library.<n>We also show that SkillFoundry can design new task-specific skills on demand for concrete scientific objectives.
  arXiv  Detail & Related papers  (2026-04-05T05:02:18Z)
• Automating Skill Acquisition through Large-Scale Mining of Open-Source Agentic Repositories: A Framework for Multi-Agent Procedural Knowledge Extraction [20.467661664708086]
  This report investigates a framework for automated acquisition of high-quality agent skills through mining of open-source repositories.<n>We focus on the extraction of visualization and educational capabilities from state-of-the-art systems including TheoremExplainAgent and Code2Video.<n>We demonstrate that systematic extraction from agentic repositories, combined with rigorous security governance and multi-dimensional evaluation metrics, enables scalable acquisition of procedural knowledge.
  arXiv  Detail & Related papers  (2026-03-12T11:10:13Z)
• SkillCraft: Can LLM Agents Learn to Use Tools Skillfully? [67.69996753743129]
  We introduce SkillCraft, a benchmark explicitly stress-test agent ability to form and reuse higher-level tool compositions.<n> SkillCraft features realistic, highly compositional tool-use scenarios with difficulty scaled along both quantitative and structural dimensions.<n>We propose a lightweight evaluation protocol that enables agents to auto-compose atomic tools into executable Skills, cache and reuse them inside and across tasks.
  arXiv  Detail & Related papers  (2026-02-28T15:44:31Z)
• PolySkill: Learning Generalizable Skills Through Polymorphic Abstraction [20.687269802717893]
  We introduce PolySkill, a new framework that enables agents to learn generalizable and compositional skills.<n> Experiments show that our method improves skill reuse by 1.7x on seen websites.<n>By enabling the agent to identify and refine its own goals, the PolySkill enhances the agent's ability to learn a better curriculum.
  arXiv  Detail & Related papers  (2025-10-17T17:56:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.