SKT: Skill-Use Training at Scale via Verified Synthetic Data Generation

• URL: http://arxiv.org/abs/2608.02287v1
• Date: Mon, 03 Aug 2026 14:18:00 GMT
• Title: SKT: Skill-Use Training at Scale via Verified Synthetic Data Generation
• Abstract summary: We introduce SKT, a verified data synthesis pipeline to improve skill-use capabilities.<n>Using 2,000 public skills, SKT produces 4,000 task packages and 27,164 verified trajectories.<n>Experiments across diverse models, benchmarks, and agent harnesses show that supervised fine-tuning on SKT-generated trajectories consistently improves skill-use performance.
• Score: 34.60026807957888
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Agent skills have become an important mechanism for equipping language-model agents with reusable procedural knowledge. However, providing skills alone does not guarantee that current models can effectively identify, apply, and coordinate them. To improve skill-use capabilities, we introduce SKT, a verified data synthesis pipeline that constructs skill-grounded tasks and executable trajectories from large collections of agent skills. SKT selects suitable single-skill and multi-skill configurations, synthesizes tasks through rule-based and agent-based verification with feedback-guided repair, and retains only successful trajectories that substantially use every required skill. Using 2,000 public skills, SKT produces 4,000 task packages and 27,164 verified trajectories. Based on the same pipeline and a disjoint test pool, we further construct SkillEval, a held-out executable benchmark for evaluating skill use. Experiments across diverse models, benchmarks, and agent harnesses show that supervised fine-tuning on SKT-generated trajectories consistently improves skill-use performance. Verification ablations, cross-harness evaluation, and scaling experiments further demonstrate that these gains depend on high-quality supervision, extend beyond a single agent interface, and increase with broader skill coverage. Together, these results establish verified data synthesis as an effective and scalable approach for skill-use training.

Related papers

• SkillCommit: Evolving Agent Skills through Behaviorally Validated Scope Expansion [15.498568310060229]
  We propose SkillCommit, an online skill evolution framework that transforms experience into a hierarchical library of reusable skills.<n>As related skills accumulate, SkillCommit abstracts those sharing a common behavioral mechanism into higher-level skills.<n> Experiments on RuleArena, OpenExempt and KOR-Bench demonstrate that SkillCommit consistently improves agent performance across diverse domains.
  arXiv  Detail & Related papers  (2026-08-15T11:03:07Z)
• A Framework for Evaluating Agentic Skills at Scale [0.4509560087514856]
  Agent skills are structured, reusable knowledge artifacts that augment agent capabilities.<n>No reusable methodology exists for evaluating an individual skill.<n>We present an evaluation framework that lets a skill author construct realistic tasks.
  arXiv  Detail & Related papers  (2026-06-16T11:46:56Z)
• SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization [99.18305770138677]
  Agent skills are reusable strategies that guide agent reasoning and action.<n>Current skill construction methods treat the problem as one-shot extraction.<n>We introduce SkillComposer, a framework that decomposes skill construction into three learnable operations.
  arXiv  Detail & Related papers  (2026-06-04T12:16:08Z)
• OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents [15.598856888948093]
  We present textscOpenSkillEval, an automatic evaluation framework for both skill-augmented agent systems and the skills themselves.<n>Instead of relying on static benchmarks, textscOpenSkillEval automatically constructs realistic task instances from evolving real-world artifacts.<n>Using more than 600 dynamically generated task instances and 30 open-source skills, we conduct a systematic evaluation of state-of-the-art models and agent frameworks.
  arXiv  Detail & Related papers  (2026-05-22T14:09:41Z)
• Skill-R1: Agent Skill Evolution via Reinforcement Learning [84.35984979949502]
  Skill-R1 is a reinforcement learning framework for instance-level recurrent skill optimization from verifiable rewards.<n>It preserves black-box compatibility with both open- and closed-source models while making adaptation substantially cheaper than model-level updates.
  arXiv  Detail & Related papers  (2026-05-10T06:19:15Z)
• SkillGen: Verified Inference-Time Agent Skill Synthesis [60.927977774369516]
  SkillGen is a framework that synthesizes a single auditable skill from trajectories generated by a base agent.<n>It identifies reusable success patterns, recurring failure modes, and behaviors that appear in nearby successes but are missing from failures.<n>A key novelty in SkillGen is that we model agent skills as interventions to empirically verify the net effect of skills on the overall performance.
  arXiv  Detail & Related papers  (2026-05-09T19:24:11Z)
• CoVe: Training Interactive Tool-Use Agents via Constraint-Guided Verification [17.56502992098113]
  We introduce textbfCoVe (textbfConstraint-textbfVerification), a post-training data synthesis framework designed for training interactive tool-use agents.<n>CoVe begins by defining explicit task constraints, which serve a dual role: they guide the generation of complex trajectories and act as deterministic verifiers for assessing trajectory quality.
  arXiv  Detail & Related papers  (2026-03-02T14:56:35Z)
• SkillCraft: Can LLM Agents Learn to Use Tools Skillfully? [67.69996753743129]
  We introduce SkillCraft, a benchmark explicitly stress-test agent ability to form and reuse higher-level tool compositions.<n> SkillCraft features realistic, highly compositional tool-use scenarios with difficulty scaled along both quantitative and structural dimensions.<n>We propose a lightweight evaluation protocol that enables agents to auto-compose atomic tools into executable Skills, cache and reuse them inside and across tasks.
  arXiv  Detail & Related papers  (2026-02-28T15:44:31Z)
• AdvKT: An Adversarial Multi-Step Training Framework for Knowledge Tracing [64.79967583649407]
  Knowledge Tracing (KT) monitors students' knowledge states and simulates their responses to question sequences.<n>Existing KT models typically follow a single-step training paradigm, which leads to significant error accumulation.<n>We propose a novel Adversarial Multi-Step Training Framework for Knowledge Tracing (AdvKT) which focuses on the multi-step KT task.
  arXiv  Detail & Related papers  (2025-04-07T03:31:57Z)
• Design of Negative Sampling Strategies for Distantly Supervised Skill Extraction [19.43668931500507]
  We propose an end-to-end system for skill extraction, based on distant supervision through literal matching. We observe that using the ESCO taxonomy to select negative examples from related skills yields the biggest improvements. We release the benchmark dataset for research purposes to stimulate further research on the task.
  arXiv  Detail & Related papers  (2022-09-13T13:37:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.