SkillReason: Reasoning-Enhanced Agent Skill Retrieval for Implicit User Requests

• URL: http://arxiv.org/abs/2608.08640v1
• Date: Sun, 09 Aug 2026 11:19:20 GMT
• Title: SkillReason: Reasoning-Enhanced Agent Skill Retrieval for Implicit User Requests
• Abstract summary: We introduce SkillReason-Bench, a large-scale cross-domain benchmark containing 3,729 queries and a retrieval corpus of 61,228 skills spanning nine domains.<n>We propose SkillRea- son, a two-stage framework that uses chain-of-thought rea- soning as training-time supervision for skill retrieval.<n>Experiments on SkillReason-Bench, SkillRet, and SRA-Bench show that Skill- Reason achieves state-of-the-art performance across all three benchmarks.
• Score: 13.420494836976033
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model agents increasingly rely on reusable skills to extend their capabilities beyond parametric knowl- edge. However, retrieving the appropriate skill from a large- scale library remains challenging because realistic user re- quests are often concise and underspecified, stating only the task goal while leaving the required capabilities and execu- tion steps implicit. Existing benchmarks provide limited cov- erage of such requests. To address this gap, we introduce SkillReason-Bench, a large-scale cross-domain benchmark containing 3,729 queries and a retrieval corpus of 61,228 skills spanning nine domains. We further propose SkillRea- son, a two-stage framework that uses chain-of-thought rea- soning as training-time supervision for skill retrieval. In Stage I, capability reasoning traces generated by a stronger teacher provide explicit supervision through contrastive learning, re- trieval distribution alignment, and language modeling, en- couraging the retriever to internalize capability reasoning in its query representation. In Stage II, a retrieval-guided GRPO objective encourages the model to explore reasoning trajecto- ries better suited to its own capabilities and more effective for retrieval. At inference, SkillReason directly encodes the orig- inal query without autoregressive CoT generation, preserv- ing efficient query-only retrieval. Extensive experiments on SkillReason-Bench, SkillRet, and SRA-Bench show that Skill- Reason achieves state-of-the-art performance across all three benchmarks, demonstrating that reasoning-enhanced training better bridges the semantic gap between high-level task goals and skill capabilities.

Related papers

• Imagine Before Retrieval: Prospective Skill Retrieval for LLM Agents [19.57498769577715]
  We propose SkillDreamer, a novel framework to alleviate the negative impact of Query--Skill Misalignment (QSM) problem.<n>In brief, SkillDreamer infers the capabilities necessary for task execution, then imagines how to realize these capabilities by generating pseudo skills.<n> Extensive experiments on SkillRet and SkillUsage prove the effectiveness of SkillDreamer in both skill retrieval and end-to-end task execution.
  arXiv  Detail & Related papers  (2026-08-28T13:19:50Z)
• Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents [67.97365535829098]
  Learning from experience is critical for developing capable, self-improving large language model (LLM) agents.<n>We propose Evo-Harness, which distills noisy, single-shot executions into reusable skill harnesses for cross-domain and topic-level adaptation.<n>Our analysis demonstrates the effectiveness of Evo-Harness and provides a principled understanding of how LLM agents can effectively learn on the fly.
  arXiv  Detail & Related papers  (2026-08-15T06:43:56Z)
• Search2Skill: Skill Distillation Beyond Knowledge Boundaries Via Rubric-Based Reinforcement Learning [38.331356573409884]
  We propose a novel framework, Search2Skill, that automatically identifies the agent's capability gaps, searches external sources to address them, and distills the retrieved evidence into structured, reusable skills.<n>Experiments on eight expert-level domains from three benchmarks show that Search2Skill consistently outperforms both search-augmented and trajectory-based skill-learning baselines.
  arXiv  Detail & Related papers  (2026-08-05T15:09:07Z)
• Rethinking Self-Evolution: A Constrained Exploration-Exploitation Process for Mitigating Skill Overfitting [20.811638832981178]
  Large language model (LLM) agents need to accumulate and reuse experience from past interactions.<n>Data-driven skill optimization is prone to overfitting to the limited trajectories collected from real environments.<n>We propose SkillBoost, a three-stage framework that mitigates both risks.
  arXiv  Detail & Related papers  (2026-07-29T09:05:40Z)
• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills [53.15216425100295]
  We introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a skill controller.<n>The proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library.<n>This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration.
  arXiv  Detail & Related papers  (2026-07-24T17:59:22Z)
• SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization [99.18305770138677]
  Agent skills are reusable strategies that guide agent reasoning and action.<n>Current skill construction methods treat the problem as one-shot extraction.<n>We introduce SkillComposer, a framework that decomposes skill construction into three learnable operations.
  arXiv  Detail & Related papers  (2026-06-04T12:16:08Z)
• SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agents [8.628588436048952]
  We introduce SkillRet, a large-scale benchmark for skill retrieval in LLM agents.<n>It provides 63,259 training samples and 4,997 evaluation queries with disjoint skill pools.<n>Off-the-shelf models struggle on realistic large-scale skill libraries, and prior skill-retrieval models still leave substantial headroom.
  arXiv  Detail & Related papers  (2026-05-07T06:18:11Z)
• From Context to Skills: Can Language Models Learn from Context Skillfully? [56.5857437218136]
  Ctx2Skill is a self-evolving framework that autonomously discovers, refines, and selects context-specific skills.<n>It can be plugged into any language model to obtain better context learning capability.
  arXiv  Detail & Related papers  (2026-04-30T09:53:15Z)
• Skill Retrieval Augmentation for Agentic AI [28.71123853744993]
  In existing agent systems, the dominant strategy for incorporating skills is to enumerate available skills within the context window.<n>This paper formulates Skill Retrieval Augmentation (SRA), a new paradigm in which agents dynamically retrieve, incorporate, and apply relevant skills from large external skill corpora on demand.<n>SRA-Bench is the first benchmark for evaluation of the full SRA pipeline, covering skill retrieval, skill incorporation, and end-task execution.
  arXiv  Detail & Related papers  (2026-04-27T15:19:59Z)
• How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings [29.3744517703302]
  We study the first comprehensive study of skill utility under progressively challenging realistic settings.<n>Our findings reveal that performance gains degrade consistently as settings become more realistic.<n>We show that query-specific refinement substantially recovers lost performance when the initial skills are of reasonable relevance and quality.
  arXiv  Detail & Related papers  (2026-04-06T00:10:30Z)
• DeepRAG: Thinking to Retrieve Step by Step for Large Language Models [92.87532210660456]
  We propose DeepRAG, a framework that models retrieval-augmented reasoning as a Markov Decision Process (MDP)<n>By iteratively decomposing queries, DeepRAG dynamically determines whether to retrieve external knowledge or rely on parametric reasoning at each step.<n> Experiments show that DeepRAG improves retrieval efficiency and boosts answer accuracy by 26.4%, demonstrating its effectiveness in enhancing retrieval-augmented reasoning.
  arXiv  Detail & Related papers  (2025-02-03T08:22:45Z)
• Skill-Based Reinforcement Learning with Intrinsic Reward Matching [77.34726150561087]
  We present Intrinsic Reward Matching (IRM), which unifies task-agnostic skill pretraining and task-aware finetuning. IRM enables us to utilize pretrained skills far more effectively than previous skill selection methods.
  arXiv  Detail & Related papers  (2022-10-14T00:04:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.