Daily Paper Briefing

Abstract Overview

This paper studies in-policy skill selection for long-horizon agents that must choose which skill file to read from a candidate slate during an episode. The authors argue that standard outcome-only policy gradients fail to train this choice because the few tokens that name the selected skill receive very little credit and often inherit the wrong sign when later execution fails, a phenomenon they call selector credit starvation. To address this, SkillGate partitions training credit into two disjoint channels: sequence-level outcome credit is applied only to execution tokens, while an action-local selector advantage is applied only to the skill-naming tokens. The method is evaluated on five agent benchmarks with 16-candidate slates, alongside controlled comparisons to outcome-only RL, supervised selection, and external routing baselines.

Novelty

The paper's main novelty is to treat mid-episode skill selection as a separate trainable decision inside a single policy-gradient update, rather than leaving it under a broadcast trajectory-level reward. It also contributes an empirical diagnosis of selector credit starvation from audited training trajectories and a token-level masking and reweighting scheme that makes selection credit length-invariant and disjoint from execution credit.

Results

On the five-benchmark mixed-slate evaluation, SkillGate achieves 53.2% overall trial success, compared with 47.0% for the matched outcome-only RL baseline and 40.8% for the shared SFT initialization. It also increases oracle-skill reading and reduces misleading-skill exposure, cutting exposure to misleading candidates by approximately two thirds while reading fewer skills overall.

Key Points

1. The paper identifies selector credit starvation as a structural reason outcome-only RL struggles to train skill choice in long-horizon trajectories.
2. SkillGate assigns outcome credit only to execution tokens and selector credit only to skill-identity tokens, with explicit masking and normalization to keep the two channels disjoint.
3. Empirically, SkillGate delivers the strongest 9B-scale results across five agent benchmarks under 16-candidate slates, improving task success and skill-selection behavior relative to controlled baselines.

References

• arXiv: https://arxiv.org/abs/2608.18852v1
• Fugu-MT: https://fugumt.com/fugumt/paper_check/2608.18852v1
• Hugging Face Papers: https://huggingface.co/papers/2608.18852
• GitHub: https://github.com/DeepExperience/SkillGate
• Hugging Face: https://huggingface.co/simonlqy/SkillGate-9B