SKILLC: Learning Autonomous Skill Internalization in LLM Agents via Contrastive Credit Assignment

• URL: http://arxiv.org/abs/2605.27899v1
• Date: Wed, 27 May 2026 03:21:19 GMT
• Title: SKILLC: Learning Autonomous Skill Internalization in LLM Agents via Contrastive Credit Assignment
• Abstract summary: We propose SkillC, a framework based on Contrastive Skill Credit Assignment (CSCA) that converts this contrast into a direct learning signal for internalization.<n>textscSkillC samples paired skill-injected and skill-free rollouts for tasks from active skill types within the same policy update.<n>A smoothed validation-level signal further drives an adaptive curriculum over attribution strength, rollout allocation, and monotonic active-set pruning.
• Score: 4.336665585098371
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Structured skill prompts improve exploration in long-horizon agentic reinforcement learning (RL). Skill-augmented RL methods retain external skills at inference, while skill-internalization RL methods withdraw them during training to enable autonomous performance. However, existing internalization approaches only use skill-helpfulness contrast for curriculum control, leaving the policy update unchanged and unable to distinguish skill-dependent from autonomous success. We propose SkillC, a framework based on Contrastive Skill Credit Assignment (CSCA) that converts this contrast into a direct learning signal for internalization. \textsc{SkillC} samples paired skill-injected and skill-free rollouts for tasks from active skill types within the same policy update, and injects their task-level contrast into optimization via a dual-stream advantage estimator that preserves global ranking while applying a one-sided correction toward skill-free success. A smoothed validation-level signal further drives an adaptive curriculum over attribution strength, rollout allocation, and monotonic active-set pruning. Experiments on ALFWorld and WebShop show that, without runtime skill access, SkillC surpasses the strongest prior skill-internalization RL baseline by 5.5\% and 4.4\%, respectively, while remaining competitive with skill-augmented RL methods.

Related papers

• Coalition-Aware Skill Reliability for Self-Evolving Agents [49.43736235959317]
  We investigate whether accumulated skills in an agent's skill bank actually make positive mechanistic contributions.<n>We show that coalition pollution, where bank-level gains conceal negative coalition-level skill contributions, and cross-domain utility reversal, where source-beneficial skills reverse their effects after transfer.<n>These findings motivate two reliability interventions: coalition-aware skill selection during skill accumulation and label-free skill masking after transfer.
  arXiv  Detail & Related papers  (2026-08-23T21:35:22Z)
• AUSO: Action-Level Unified Skill Optimization from Internalization to Utilization [23.057686255420546]
  We introduce AUSO, which skill learning and skill use through a progressive, action-aware optimization process.<n>At the beginning of training, AUSO jointly learns from teacher guidance and environmental outcomes.<n>As the policy matures, AUSO evaluates each sampled action under both skill-conditioned and skill-free contexts.
  arXiv  Detail & Related papers  (2026-08-21T16:53:27Z)
• Bidirectional Context Self-Distillation for Reinforcement Learning of Skill-Based LLM Agents [11.967142343676171]
  We propose BCSD (Bi Context Self-Distillation), a framework that combines self-distillation and reinforcement learning to train agents to use external skills more effectively.<n> Experiments on ALFWorld and WebShop demonstrate that BCSD achieves the strongest overall performance across model scales.
  arXiv  Detail & Related papers  (2026-08-10T12:53:06Z)
• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills [53.15216425100295]
  We introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a skill controller.<n>The proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library.<n>This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration.
  arXiv  Detail & Related papers  (2026-07-24T17:59:22Z)
• ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RL [24.65745354725497]
  We introduce ReSkill, an RL-in-the-loop skill creation framework that reconciles skill evolution with policy learning.<n>ReSkill exploits the group-wise structure of GRPO to embed three mechanisms with only marginal additional overhead.<n>Across several domains, ReSkill consistently outperforms existing memory and skill-based RL methods.
  arXiv  Detail & Related papers  (2026-06-01T03:12:05Z)
• Skill0.5: Joint Skill Internalization and Utilization for Out-of-Distribution Generalization in Agentic Reinforcement Learning [25.966432263031397]
  Skill0.5 is a novel agentic RL framework that explicitly differentiates skill treatments by combining general skill internalization with task-specific skill utilization.<n>It internalizes general skills via privileged distillation to build a cognitive foundation for hard tasks, while using diagnostic probing on easy tasks to penalize shortcuts and enforce specific skill utilization.<n> Experiments on ALFWorld and WebShop demonstrate that Skill0.5 outperforms both memory-based and skill-based RL baselines.
  arXiv  Detail & Related papers  (2026-05-27T12:54:33Z)
• Harnessing LLM Agents with Skill Programs [58.356514745548026]
  We introduce HASP, a new framework that upgrades skills into executable Program Functions (PFs)<n>PFs act as executable guardrails that activate on failure-prone states and modify the next action or inject corrective context.<n> HASP drives substantial gains compared to both training-free and training-based methods on web-search, math reasoning, and coding tasks.
  arXiv  Detail & Related papers  (2026-05-18T01:35:11Z)
• Dynamic Skill Lifecycle Management for Agentic Reinforcement Learning [11.76119198695392]
  Large language model agents increasingly rely on external skills to solve complex tasks.<n>Existing methods assume external skills either accumulate as persistent guidance or internalized into the policy, leading to zero-skill inference.<n>We argue this assumption is overly restrictive, since with limited parametric capacity and uneven marginal contribution across skills, the optimal active skill set is non-monotonic, task- and stage-dependent.
  arXiv  Detail & Related papers  (2026-05-11T17:55:13Z)
• Skill-R1: Agent Skill Evolution via Reinforcement Learning [84.35984979949502]
  Skill-R1 is a reinforcement learning framework for instance-level recurrent skill optimization from verifiable rewards.<n>It preserves black-box compatibility with both open- and closed-source models while making adaptation substantially cheaper than model-level updates.
  arXiv  Detail & Related papers  (2026-05-10T06:19:15Z)
• SkillOS: Learning Skill Curation for Self-Evolving Agents [67.94374107466957]
  We propose SkillOS, an experience-driven RL training recipe for learning skill curation in self-evolving agents.<n>SkillOS pairs a frozen agent executor that retrieves and applies skills with a trainable skill curator that updates an external SkillRepo from accumulated experience.<n>We show that SkillOS consistently outperforms memory-free and strong memory-based baselines in both effectiveness and efficiency.
  arXiv  Detail & Related papers  (2026-05-07T17:31:50Z)
• SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization [61.37478254738943]
  Inference-time skill augmentation is fundamentally limited.<n>We introduce SKILL0, an in-context reinforcement learning framework designed for skill internalization.<n>SKILL0 achieves substantial improvements over the standard RL baseline.
  arXiv  Detail & Related papers  (2026-04-02T17:03:05Z)
• Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learning [6.959686714606018]
  Self-Improving Skill Learning (SISL) performs self-guided skill refinement using decoupled high-level and skill improvement policies.<n>SISL consistently outperforms other skill-based meta-RL methods on diverse long-horizon tasks.
  arXiv  Detail & Related papers  (2025-02-06T03:28:45Z)
• Skill-Based Reinforcement Learning with Intrinsic Reward Matching [77.34726150561087]
  We present Intrinsic Reward Matching (IRM), which unifies task-agnostic skill pretraining and task-aware finetuning. IRM enables us to utilize pretrained skills far more effectively than previous skill selection methods.
  arXiv  Detail & Related papers  (2022-10-14T00:04:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.