Harnessing LLM Agents with Skill Programs

• URL: http://arxiv.org/abs/2605.17734v1
• Date: Mon, 18 May 2026 01:35:11 GMT
• Title: Harnessing LLM Agents with Skill Programs
• Abstract summary: We introduce HASP, a new framework that upgrades skills into executable Program Functions (PFs)<n>PFs act as executable guardrails that activate on failure-prone states and modify the next action or inject corrective context.<n> HASP drives substantial gains compared to both training-free and training-based methods on web-search, math reasoning, and coding tasks.
• Score: 58.356514745548026
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Equipping LLM agents with reusable skills derived from past experience has become a popular and successful approach for tackling complex and long-horizon tasks. However, such lessons are often encoded as textual guidance that remains largely advisory, lacking explicit mechanisms for when and how to intervene in the agent loop. To bridge the gap, we introduce HASP(Harnessing LLM Agents with Skill Programs), a new framework that upgrades skills into executable Program Functions (PFs). Rather than offering passive advice, PFs act as executable guardrails that activate on failure-prone states and modify the next action or inject corrective context. HASP is highly modular: it can be applied at inference time for direct agent-loop intervention, during post-training to provide structured supervision, or for self-improvement by evolving validated, teacher-reviewed PFs. Empirically, HASP drives substantial gains compared to both training-free and training-based methods on web-search, math reasoning, and coding tasks. For example, on web-search reasoning, inference-time PFs alone improve the average performance by 25% compared to (multi-loop) ReAct Agent, while post-training and controlled evolution achieve a 30.4% gain over Search-R1. To provide deeper insights into HASP, our mechanism analysis reveals how PFs trigger and intervene, how skills are internalized, and the requirement for stable skill library evolution.

Related papers

• Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents [67.97365535829098]
  Learning from experience is critical for developing capable, self-improving large language model (LLM) agents.<n>We propose Evo-Harness, which distills noisy, single-shot executions into reusable skill harnesses for cross-domain and topic-level adaptation.<n>Our analysis demonstrates the effectiveness of Evo-Harness and provides a principled understanding of how LLM agents can effectively learn on the fly.
  arXiv  Detail & Related papers  (2026-08-15T06:43:56Z)
• Agentic Reinforcement Learning with Self-Distilled Reward Shaping [12.091871903119698]
  Agentic reinforcement learning enables LLM agents to learn through interaction, but sparse trajectory-level rewards reveal success without identifying which intermediate decisions deserve credit.<n>We introduce Agentic Reinforcement Learning with Self-Distilled Reward Shaping, a framework for constructing return-associated token-level credit for language agents.<n>Experiments show that ADRS consistently improves performance on long-horizon tasks, with gains persisting across RL backbones, reduced-data settings, unseen tasks, and extended training.
  arXiv  Detail & Related papers  (2026-08-04T06:56:47Z)
• SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning [38.54413500261524]
  Outcome-based reinforcement learning (RL) provides a practical optimization paradigm, but its sparse trajectory-level rewards offer limited guidance on intermediate decisions.<n>We propose SEED (SElf-Evolving On-Policy Distillation), a self-evolving framework that converts completed on-policy trajectories into training-time hindsight skills.
  arXiv  Detail & Related papers  (2026-07-16T09:57:18Z)
• Adapting Generalist Robot Policies with Semantic Reinforcement Learning [59.009122725071386]
  Generalist robot policies learn a diverse repertoire of behaviors from large-scale pretraining.<n>We propose Semantic Action Reinforcement Learning (SARL) to learn to optimize this prompt space through online interaction.<n>We show SARL unlocks fundamentally new capabilities -- adapting VLA behavior to solve complex, long-horizon tasks.
  arXiv  Detail & Related papers  (2026-06-30T17:00:33Z)
• Self-evolving LLM agents with in-distribution Optimization [60.05867547965365]
  Large Language Models (LLMs) have recently emerged as powerful controllers for interactive agents in complex environments.<n>We propose Q-Evolve, a self-evolving framework for LLM agents that unifies automatic process-reward labeling and policy learning.<n>We evaluate our method on AlfWorld, WebShop, and ScienceWorld, showing Q-Evolve outperforms strong baselines in sample efficiency, robustness, and overall task performance.
  arXiv  Detail & Related papers  (2026-06-05T15:09:52Z)
• PRO-CUA: Process-Reward Optimization for Computer Use Agents [25.2822788411397]
  PRO-CUA is a process-reward optimization framework for training computer use agents.<n>It decouples on-policy environment interaction from policy optimization.<n>It reduces distribution shift by training on the agent's own execution states.
  arXiv  Detail & Related papers  (2026-05-27T21:28:26Z)
• Agentic Reinforcement Learning with Implicit Step Rewards [92.26560379363492]
  Large language models (LLMs) are increasingly developed as autonomous agents using reinforcement learning (agentic RL)<n>We introduce implicit step rewards for agentic RL (iStar), a general credit-assignment strategy that integrates seamlessly with standard RL algorithms.<n>We evaluate our method on three challenging agent benchmarks, including WebShop and VisualSokoban, as well as open-ended social interactions with unverifiable rewards in SOTOPIA.
  arXiv  Detail & Related papers  (2025-09-23T16:15:42Z)
• Agentic Reinforced Policy Optimization [66.96989268893932]
  Large-scale reinforcement learning with verifiable rewards (RLVR) has demonstrated its effectiveness in harnessing the potential of large language models (LLMs) for single-turn reasoning tasks.<n>Current RL algorithms inadequately balance the models' intrinsic long-horizon reasoning capabilities and their proficiency in multi-turn tool interactions.<n>We propose Agentic Reinforced Policy Optimization (ARPO), a novel agentic RL algorithm tailored for training multi-turn LLM-based agents.
  arXiv  Detail & Related papers  (2025-07-26T07:53:11Z)
• From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning [62.54484062185869]
  We introduce StepAgent, which utilizes step-wise reward to optimize the agent's reinforcement learning process.<n>We propose implicit-reward and inverse reinforcement learning techniques to facilitate agent reflection and policy adjustment.
  arXiv  Detail & Related papers  (2024-11-06T10:35:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.