Progressive Agent Skill Generation via Reinforcement Learning

• URL: http://arxiv.org/abs/2608.01678v1
• Date: Mon, 03 Aug 2026 04:14:59 GMT
• Title: Progressive Agent Skill Generation via Reinforcement Learning
• Abstract summary: We propose Skill-$$, a reinforcement learning method for progressively generating high-quality agent skills.<n>Specifically, we formulate skill generation as a sequential editing process that decomposes skill construction into individually evaluable edits.<n>Experiments show that Skill-$$ generates more effective skills than methods based on pipeline-style consolidation.
• Score: 29.782307460602876
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing skill generation methods largely rely on heuristics or pipeline-style consolidation, which must be specially designed for different evidence sources. In contrast, learning-based approaches offer a more unified way to model skill generation across heterogeneous sources. However, learning-based skill generation remains challenging because skills lack a natural supervision signal based on relevance or correctness; their value can largely be determined only by whether they improve the behavior of the agent on downstream tasks. To address this challenge, we propose Skill-$α$, a reinforcement learning method for progressively generating high-quality agent skills. Specifically, we formulate skill generation as a sequential editing process that decomposes skill construction into individually evaluable edits, and introduce a novel rollback reward that evaluates each edit by comparing downstream execution under the original and edited skills on an anchored query. Extensive experiments show that Skill-$α$ generates more effective skills than methods based on heuristics or pipelines in both document-to-skill and experience-to-skill settings. Under the main GPT-4o worker, Skill-$α$ improves average downstream success rates over the strongest skill-generation baseline by 3.3 points on CL-Bench and 6.7 points on tau2-bench. Further ablations validate the importance of rollback reward and progressive generation.

Related papers

• SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization [99.18305770138677]
  Agent skills are reusable strategies that guide agent reasoning and action.<n>Current skill construction methods treat the problem as one-shot extraction.<n>We introduce SkillComposer, a framework that decomposes skill construction into three learnable operations.
  arXiv  Detail & Related papers  (2026-06-04T12:16:08Z)
• SkillRevise: Improving LLM-Authored Agent Skills via Trace-Conditioned Skill Revision [41.562357872885826]
  We propose SkillRevise, an execution-grounded framework designed to iteratively refine agent skills.<n>SkillRevise diagnoses skill defects from execution evidence, retrieves relevant repair principles from a general memory, and applies execution-anchored edits.<n>It substantially outperforms one-shot baselines, improving the base agent's success rate on SkillsBench from 36.05% to 61.63%.
  arXiv  Detail & Related papers  (2026-05-31T10:19:13Z)
• Skill-R1: Agent Skill Evolution via Reinforcement Learning [84.35984979949502]
  Skill-R1 is a reinforcement learning framework for instance-level recurrent skill optimization from verifiable rewards.<n>It preserves black-box compatibility with both open- and closed-source models while making adaptation substantially cheaper than model-level updates.
  arXiv  Detail & Related papers  (2026-05-10T06:19:15Z)
• SkillGen: Verified Inference-Time Agent Skill Synthesis [60.927977774369516]
  SkillGen is a framework that synthesizes a single auditable skill from trajectories generated by a base agent.<n>It identifies reusable success patterns, recurring failure modes, and behaviors that appear in nearby successes but are missing from failures.<n>A key novelty in SkillGen is that we model agent skills as interventions to empirically verify the net effect of skills on the overall performance.
  arXiv  Detail & Related papers  (2026-05-09T19:24:11Z)
• SkillMaster: Toward Autonomous Skill Mastery in LLM Agents [27.651128308229378]
  We propose SkillMaster, a training framework that teaches agents to create new skills, refine existing skills, and select accumulated skills during task solving.<n>First, we train agents through trajectory-informed skill review, teaching agents to propose, update, or retain skills based on evidence from completed episodes.<n>Second, each candidate skill edit is designed to be evaluated by its counterfactual utility on related probe tasks, providing a direct learning signal for training skill-editing decisions.<n>Third, we introduce DualAdv-GRPO, which separately estimates advantages for task-solving actions and skill-editing decisions, stabilizing joint training across task solving and
  arXiv  Detail & Related papers  (2026-05-09T05:03:00Z)
• SkillLens: Adaptive Multi-Granularity Skill Reuse for Cost-Efficient LLM Agents [6.293920920196533]
  SkillLens is a hierarchical skill-evolution framework that organizes skills into a four-layer graph of policies, strategies, procedures, and primitives.<n>It retrieves semantically relevant skill seeds, expands them through degree-corrected random walk over the skill graph, and then uses a verifier to decide whether each visited unit should be accepted, decomposed, rewritten, or skipped.<n>Across MuLocbench and ALFWorld, SkillLens consistently improves over strong skill-based baselines.
  arXiv  Detail & Related papers  (2026-05-08T18:48:04Z)
• SkillOS: Learning Skill Curation for Self-Evolving Agents [67.94374107466957]
  We propose SkillOS, an experience-driven RL training recipe for learning skill curation in self-evolving agents.<n>SkillOS pairs a frozen agent executor that retrieves and applies skills with a trainable skill curator that updates an external SkillRepo from accumulated experience.<n>We show that SkillOS consistently outperforms memory-free and strong memory-based baselines in both effectiveness and efficiency.
  arXiv  Detail & Related papers  (2026-05-07T17:31:50Z)
• EvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification [85.3692584167951]
  Anthropic proposes the concept of skills for LLM agents to tackle multi-step professional tasks.<n>A tool is a single, self-contained function, whereas a skill is a structured bundle of interdependent multi-file artifacts.<n>EvoSkills is a self-evolving skills framework that enables agents to autonomously construct complex, multi-file skill packages.
  arXiv  Detail & Related papers  (2026-04-02T06:43:20Z)
• Skill-Based Reinforcement Learning with Intrinsic Reward Matching [77.34726150561087]
  We present Intrinsic Reward Matching (IRM), which unifies task-agnostic skill pretraining and task-aware finetuning. IRM enables us to utilize pretrained skills far more effectively than previous skill selection methods.
  arXiv  Detail & Related papers  (2022-10-14T00:04:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.