SkillRouter: Retrieve-and-Rerank Skill Selection for LLM Agents at Scale

• URL: http://arxiv.org/abs/2603.22455v1
• Date: Mon, 23 Mar 2026 18:23:59 GMT
• Title: SkillRouter: Retrieve-and-Rerank Skill Selection for LLM Agents at Scale
• Abstract summary: Skill routing is needed to retrieve the most relevant skills from a large pool given a user task.<n>Despite its practical importance, skill routing remains under-explored.<n>We propose Skill, a two-stage retrieve-and-rerank pipeline totaling only 1.2B parameters.
• Score: 5.512890692955285
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As LLM agent ecosystems grow, the number of available skills (tools, plugins) has reached tens of thousands, making it infeasible to inject all skills into an agent's context. This creates a need for skill routing -- retrieving the most relevant skills from a large pool given a user task. The problem is compounded by pervasive functional overlap in community skill repositories, where many skills share similar names and purposes yet differ in implementation details. Despite its practical importance, skill routing remains under-explored. Current agent architectures adopt a progressive disclosure design -- exposing only skill names and descriptions to the agent while keeping the full implementation body hidden -- implicitly treating metadata as sufficient for selection. We challenge this assumption through a systematic empirical study on a benchmark of ~$80K skills and 75 expert-verified queries. Our key finding is that the skill body (full implementation text) is the decisive signal: removing it causes 29--44 percentage point degradation across all retrieval methods, and cross-encoder attention analysis reveals 91.7% of attention concentrating on the body field. Motivated by this finding, we propose SkillRouter, a two-stage retrieve-and-rerank pipeline totaling only 1.2B parameters (0.6B encoder + 0.6B reranker). SkillRouter achieves 74.0% top-1 routing accuracy and delivers the strongest average result among the compact and zero-shot baselines we evaluate, while remaining deployable on consumer hardware.

Related papers

• The Router Within: Eliciting Native Skill Routing from a Frozen LLM [36.31894794808421]
  Skills extend an agent's parametric knowledge, and the gain they promise rests on picking the right one.<n>We show that the frozen agent LLM already carries the routing signal in its own forward passes.<n>A glance projects the task's and each skill's mid-layer states through the two maps, the only parameters trained.<n>A verdict then resumes the shortlisted skills' forward passes and reads the model's own likelihood and yes/no judgment.
  arXiv  Detail & Related papers  (2026-09-14T17:58:27Z)
• Beyond Top-$k$ Skill Retrieval: Diversity-Aware Skill Routing for LLM Agents [79.02474149053074]
  We propose Diverse Skill Routing (DSR), a diversity-aware reranking framework.<n>DSR penalizes redundant skill overlap while reducing penalties caused only by shared query relevance.<n>On the Skill routing benchmark, DSR improves recall and full coverage over a strong pointwise reranking baseline.
  arXiv  Detail & Related papers  (2026-09-05T02:39:28Z)
• Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills [86.15447717549154]
  We present DisCo, a skill-powered research agent that creates skills and uses them during research.<n>We call this missing layer operational knowledge, the know-how that separates knowing a method from making it work.
  arXiv  Detail & Related papers  (2026-09-02T15:49:41Z)
• Generative Skill Composition for LLM Agents [58.47191648555256]
  LLM agents benefit from skills for solving complex tasks.<n>As skill libraries grow, selecting an appropriate skill composition has emerged as a central bottleneck.<n>We propose SkillComposer, which instantiates structured skill composition as task-conditioned skill sequence prediction.
  arXiv  Detail & Related papers  (2026-06-30T17:53:09Z)
• Parametric Skills [18.083331448567936]
  ParametricSkills is a framework that can convert free-form textual skills into parameters at test time, enabling context-free skill exploitation.<n>We show that, the proposed ParametricSkills averagely outperforms in-context learning by 6.44 points as judged by DeepSeek-V4-Flash.
  arXiv  Detail & Related papers  (2026-06-29T09:19:32Z)
• SkillJuror: Measuring How Agent Skill Organization Changes Runtime Behavior [23.11308948097101]
  Current benchmarks rarely distinguish what a Skill says from how it is organized.<n>We study this distinction through Progressive Disclosure, where a concise root file points agents to supporting resources on demand.<n>We present SkillJuror, a framework for evaluating Skill writing paradigms.
  arXiv  Detail & Related papers  (2026-06-10T01:11:50Z)
• SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization [99.18305770138677]
  Agent skills are reusable strategies that guide agent reasoning and action.<n>Current skill construction methods treat the problem as one-shot extraction.<n>We introduce SkillComposer, a framework that decomposes skill construction into three learnable operations.
  arXiv  Detail & Related papers  (2026-06-04T12:16:08Z)
• How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings [29.3744517703302]
  We study the first comprehensive study of skill utility under progressively challenging realistic settings.<n>Our findings reveal that performance gains degrade consistently as settings become more realistic.<n>We show that query-specific refinement substantially recovers lost performance when the initial skills are of reasonable relevance and quality.
  arXiv  Detail & Related papers  (2026-04-06T00:10:30Z)
• SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization [61.37478254738943]
  Inference-time skill augmentation is fundamentally limited.<n>We introduce SKILL0, an in-context reinforcement learning framework designed for skill internalization.<n>SKILL0 achieves substantial improvements over the standard RL baseline.
  arXiv  Detail & Related papers  (2026-04-02T17:03:05Z)
• Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills [21.971456179996093]
  We introduce Trace2Skill, a framework that mirrors how human experts author skills.<n>Instead of reacting sequentially to individual trajectories, Trace2Skill dispatches a parallel fleet of sub-agents to analyze a diverse pool of executions.<n>It extracts trajectory-specific lessons and hierarchically consolidates them into a unified, conflict-free skill directory via inductive reasoning.
  arXiv  Detail & Related papers  (2026-03-26T08:26:38Z)
• SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering? [27.583012125198568]
  SWE-Skills-Bench is the first requirement-driven benchmark that isolates the marginal utility of agent skills in real-world software engineering.<n>It pairs 49 public SWE skills with authentic GitHub repositories pinned at fixed commits and requirement documents with explicit acceptance criteria.<n>Our results show that skill injection benefits are far more limited than rapid adoption suggests.
  arXiv  Detail & Related papers  (2026-03-16T15:16:31Z)
• OSExpert: Computer-Use Agents Learning Professional Skills via Exploration [55.660669638732024]
  General-purpose computer-use agents are far less helpful than human experts.<n>We introduce a depth-first search algorithm to explore and verify an environment's unit functions.<n>The agent then exploits compositionality between unit skills to self-construct a curriculum for composite tasks.
  arXiv  Detail & Related papers  (2026-03-09T05:27:56Z)
• EvoSkill: Automated Skill Discovery for Multi-Agent Systems [6.319876096746374]
  We introduce textbfEvoSkill, a self-evolving framework that automatically discovers and refines agent skills.<n>EvoSkill analyzes execution failures, proposes new skills or edits to existing ones, and materializes them into structured, reusable skill folders.<n>We evaluate EvoSkill on two benchmarks: OfficeQA, a grounded reasoning benchmark over U.S. Treasury data, and SealQA, a noisy retrieval benchmark.
  arXiv  Detail & Related papers  (2026-03-03T09:07:22Z)
• Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale [28.43462779191672]
  AgentSkillOS is a principled framework for skill selection, orchestration, and ecosystem-level management.<n>AgentSkillOS comprises two stages: (i) Manage Skills, which organizes skills into a capability tree.<n> (ii) Solve Tasks, which retrieves, orchestrates, and executes multiple skills through DAG-based pipelines.
  arXiv  Detail & Related papers  (2026-03-02T18:46:47Z)
• SkillNet: Create, Evaluate, and Connect AI Skills [159.47504178122156]
  SkillNet is an open infrastructure designed to create, evaluate, and organize AI skills at scale.<n>Our infrastructure integrates a repository of over 200,000 skills, an interactive platform, and a versatile Python toolkit.
  arXiv  Detail & Related papers  (2026-02-26T14:24:02Z)
• SoK: Agentic Skills -- Beyond Tool Use in LLM Agents [6.356997609995175]
  Agentic systems increasingly rely on reusable procedural capabilities, textita.k.a., agentic skills, to execute long-horizon reliably.<n>This paper maps the skill layer across the full lifecycle (discovery, practice, distillation, storage, composition, evaluation, and update)<n>We analyze the security and governance implications of skill-based agents, covering supply-chain risks, prompt injection via skill payloads, and trust-tiered execution.
  arXiv  Detail & Related papers  (2026-02-24T13:11:38Z)
• SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks [61.89812116484928]
  Agent Skills are structured packages of procedural knowledge that augment LLM agents at inference time.<n>We present SkillsBench, a benchmark of 86 tasks across 11 domains paired with curated Skills and deterministic verifiers.<n>We show curated Skills raise average pass rate by 16.2 percentage points(pp), but effects vary widely by domain.<n>Self-generated Skills provide no benefit on average, showing that models cannot reliably author the procedural knowledge they benefit from consuming.
  arXiv  Detail & Related papers  (2026-02-13T07:06:06Z)
• SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning [83.98129545309277]
  We propose SkillRL, a framework that bridges the gap between raw experience and policy improvement.<n>Our approach introduces an experience-based distillation mechanism to build a hierarchical skill library SkillBank.<n> Experimental results on ALF, WebShop and seven search-augmented tasks demonstrate that SkillRL achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-02-09T03:17:17Z)
• WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning [73.91893534088798]
  WebSailor is a complete post-training methodology designed to instill this crucial capability.<n>Our approach involves generating novel, high-uncertainty tasks through structured sampling and information obfuscation.<n>WebSailor significantly outperforms all open-source agents in complex information-seeking tasks.
  arXiv  Detail & Related papers  (2025-09-16T17:57:03Z)
• Design of Negative Sampling Strategies for Distantly Supervised Skill Extraction [19.43668931500507]
  We propose an end-to-end system for skill extraction, based on distant supervision through literal matching. We observe that using the ESCO taxonomy to select negative examples from related skills yields the biggest improvements. We release the benchmark dataset for research purposes to stimulate further research on the task.
  arXiv  Detail & Related papers  (2022-09-13T13:37:06Z)
• Hierarchical Skills for Efficient Exploration [70.62309286348057]
  In reinforcement learning, pre-trained low-level skills have the potential to greatly facilitate exploration. Prior knowledge of the downstream task is required to strike the right balance between generality (fine-grained control) and specificity (faster learning) in skill design. We propose a hierarchical skill learning framework that acquires skills of varying complexity in an unsupervised manner.
  arXiv  Detail & Related papers  (2021-10-20T22:29:32Z)
• Knowledge Integration Networks for Action Recognition [58.548331848942865]
  We design a three-branch architecture consisting of a main branch for action recognition, and two auxiliary branches for human parsing and scene recognition. We propose a two-level knowledge encoding mechanism which contains a Cross Branch Integration (CBI) module for encoding the auxiliary knowledge into medium-level convolutional features, and an Action Knowledge Graph (AKG) for effectively fusing high-level context information. The proposed KINet achieves the state-of-the-art performance on a large-scale action recognition benchmark Kinetics-400, with a top-1 accuracy of 77.8%.
  arXiv  Detail & Related papers  (2020-02-18T10:20:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.