Write, Execute, Refine: From Skill Followers to Skill Optimizers via Reinforcement Learning from Execution Feedback

• URL: http://arxiv.org/abs/2608.17587v1
• Date: Tue, 18 Aug 2026 09:52:48 GMT
• Title: Write, Execute, Refine: From Skill Followers to Skill Optimizers via Reinforcement Learning from Execution Feedback
• Abstract summary: Expert-written natural language skills can improve tool-using agents, yet agent-authored skills perform 8-11 points worse than using no skill.<n>We study how to organize execution experience from intermediate skills into training states for an executor.
• Score: 52.78692582507693
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Expert-written natural language skills can improve tool-using agents, yet agent-authored skills perform 8-11 points worse than using no skill. This gap suggests that following procedural guidance and improving it from execution evidence are distinct capabilities. Inference time loops can repair skills but do not improve the model that writes the next one. We study how to organize execution experience from intermediate skills into training states for an optimizer. We introduce WER (Write, Execute, and Refine), a multi-phase framework that trains a Skill Optimizer outside a frozen executor. The optimizer proposes skills, a frozen agent executes each repeatedly, and a programmatic verifier scores the outcomes. The scores provide relative credit and select mixed-outcome records. Matched successful and failed trajectories from these records form the next phase's refinement states, so the optimizer learns from the consequences of its earlier outputs. On BFCL v4 multi-turn and tau2-bench, WER improves average Pass@1 over the no-skill baseline by 7.80 and 3.85 points, respectively. Under an identical refinement workflow, it outperforms the same backbone without optimizer training by 9.35 and 10.29 points. The trained 4B optimizer reaches 76.63 percent on BFCL v4, outperforming all evaluated off-the-shelf general-purpose models used as skill optimizers on average.

Related papers

• COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization [55.60171725587438]
  Large language model (LLM) agents can benefit from reusable skills distilled from prior task experience.<n>Existing skill optimization methods often rely on costly execution-based evaluation and substantial task data.<n>We introduce textbfCOBRA-Skills, an efficient framework that formulates skill optimization as budgeted sequential optimization over a dynamically evolving candidate space.
  arXiv  Detail & Related papers  (2026-09-10T15:12:24Z)
• Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning [72.09826781730662]
  Long-horizon reasoning demands that the model switch between distinct skills inside a reasoning chain.<n>We introduce Skill Entropy, a measure of the difficulty of switching from one skill to another.<n>We propose Skill-Entropy RL, an RL framework where the model predicts not only the answer at each step but also the skill used to produce it.
  arXiv  Detail & Related papers  (2026-08-05T17:57:16Z)
• RLPF: Reinforcement Learning from Performance Feedback for Code Generation [44.39557506834191]
  We study how to train code agents to prefer faster correct implementations, rather than treating efficiency only as an evaluation metric.<n>We propose textbfRLPF, reinforcement learning from performance feedback, which turns execution outcomes into a staged reward.<n>Fine-tuning Qwen3-32B with RLPF on PerfCodeBench raises correct-and-runnable solutions from $11.1%$ to $54.6%$.
  arXiv  Detail & Related papers  (2026-07-29T11:39:55Z)
• Not All Skills Help: Measuring and Repairing Agent Knowledge [47.498075849595374]
  ASSAY is a framework that separates generation from curation.<n>It computes a per-skill causal attribution on a small development set.<n>It restructures the library offline, and suppresses skills with negative predicted effect for each test task.
  arXiv  Detail & Related papers  (2026-06-13T16:37:38Z)
• SkillsInjector: Dynamic Skill Context Construction for LLM Agents [16.631471381875816]
  Existing methods treat skill injection as a static step, selecting skills with fixed criteria, fixing the budget in advance, and leaving descriptions unchanged.<n>We propose SkillsInjector, a two-stage adaptive method that jointly addresses these decisions.<n>Across tau2-bench, SkillsBench, and ALFWorld, SkillsInjector achieves the highest score, improving over the strongest baseline by 3.9, 6.1, and 7.3 percentage points, respectively.
  arXiv  Detail & Related papers  (2026-05-28T11:44:32Z)
• SkillOpt: Executive Strategy for Self-Evolving Agent Skills [39.94802218531371]
  SkillOpt is a controllable text-space for agent skills.<n>A separate model turns scored rollouts into bounded add/delete/replace edits on a single skill document.<n>A learning-rate budget, rejected-edit buffer, and epoch-wise slow/meta update make skill training stable.
  arXiv  Detail & Related papers  (2026-05-22T17:59:50Z)
• Skill-R1: Agent Skill Evolution via Reinforcement Learning [84.35984979949502]
  Skill-R1 is a reinforcement learning framework for instance-level recurrent skill optimization from verifiable rewards.<n>It preserves black-box compatibility with both open- and closed-source models while making adaptation substantially cheaper than model-level updates.
  arXiv  Detail & Related papers  (2026-05-10T06:19:15Z)
• SkillGen: Verified Inference-Time Agent Skill Synthesis [60.927977774369516]
  SkillGen is a framework that synthesizes a single auditable skill from trajectories generated by a base agent.<n>It identifies reusable success patterns, recurring failure modes, and behaviors that appear in nearby successes but are missing from failures.<n>A key novelty in SkillGen is that we model agent skills as interventions to empirically verify the net effect of skills on the overall performance.
  arXiv  Detail & Related papers  (2026-05-09T19:24:11Z)
• EffiSkill: Agent Skill Based Automated Code Efficiency Optimization [18.68243664364758]
  EffiSkill is a framework for code-efficiency optimization that builds a portable optimization toolbox for LLM-based agents.<n>EffiSkill adopts a two-stage design: Stage I mines Operator and Meta Skills from large-scale slow/fast program pairs to build a skill library.<n>Stage II applies this library to unseen programs through execution-free diagnosis, skill retrieval, plan composition, and candidate generation.
  arXiv  Detail & Related papers  (2026-03-29T20:09:57Z)
• Training Learned Optimizers with Randomly Initialized Learned Optimizers [49.67678615506608]
  We show that a population of randomly learneds can be used to train themselves from scratch in an online fashion. A form of population based training is used to orchestrate this self-training. We believe feedback loops of this type will be important and powerful in the future of machine learning.
  arXiv  Detail & Related papers  (2021-01-14T19:07:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.