SkillForge: Forging Domain-Specific, Self-Evolving Agent Skills in Cloud Technical Support

• URL: http://arxiv.org/abs/2604.08618v1
• Date: Thu, 09 Apr 2026 07:11:23 GMT
• Title: SkillForge: Forging Domain-Specific, Self-Evolving Agent Skills in Cloud Technical Support
• Abstract summary: Existing skill creators lack domain grounding, producing skills poorly aligned with real-world task requirements.<n>We introduce SkillForge, a self-evolving framework that closes the end-to-end creation-refinement loop.<n>We evaluate five real-world cloud support scenarios spanning 1,883 tickets and 3,737 tasks.
• Score: 3.277125886726497
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deploying LLM-powered agents in enterprise scenarios such as cloud technical support demands high-quality, domain-specific skills. However, existing skill creators lack domain grounding, producing skills poorly aligned with real-world task requirements. Moreover, once deployed, there is no systematic mechanism to trace execution failures back to skill deficiencies and drive targeted refinements, leaving skill quality stagnant despite accumulating operational evidence. We introduce SkillForge, a self-evolving framework that closes an end-to-end creation-evaluation-refinement loop. To produce well-aligned initial skills, a Domain-Contextualized Skill Creator grounds skill synthesis in knowledge bases and historical support tickets. To enable continuous self-optimization, a three-stage pipeline -- Failure Analyzer, Skill Diagnostician, and Skill Optimizer -- automatically diagnoses execution failures in batch, pinpoints the underlying skill deficiencies, and rewrites the skill to eliminate them. This cycle runs iteratively, allowing skills to self-improve with every round of deployment feedback. Evaluated on five real-world cloud support scenarios spanning 1,883 tickets and 3,737 tasks, experiments show that: (1) the Domain-Contextualized Skill Creator produces substantially better initial skills than the generic skill creator, as measured by consistency with expert-authored reference responses from historical tickets; and (2) the self-evolution loop progressively improves skill quality from diverse starting points (including expert-authored, domain-created, and generic skills) across successive rounds, demonstrating that automated evolution can surpass manually curated expert knowledge.

Related papers

• SkillAlchemy: Open-World Agent Skill Creation [11.836010370630568]
  Agent skills are reusable procedural artifacts that extend language agents with specialized, tool conventions, and domain behaviors at inference time.<n>Create reliable skills still depends largely on human authorship, model priors, or execution traces.<n>We propose SkillAlchemy, an admission-centered framework for source-grounded skill creation.
  arXiv  Detail & Related papers  (2026-08-24T15:58:57Z)
• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills [53.15216425100295]
  We introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a skill controller.<n>The proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library.<n>This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration.
  arXiv  Detail & Related papers  (2026-07-24T17:59:22Z)
• SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization [99.18305770138677]
  Agent skills are reusable strategies that guide agent reasoning and action.<n>Current skill construction methods treat the problem as one-shot extraction.<n>We introduce SkillComposer, a framework that decomposes skill construction into three learnable operations.
  arXiv  Detail & Related papers  (2026-06-04T12:16:08Z)
• SkillRevise: Improving LLM-Authored Agent Skills via Trace-Conditioned Skill Revision [41.562357872885826]
  We propose SkillRevise, an execution-grounded framework designed to iteratively refine agent skills.<n>SkillRevise diagnoses skill defects from execution evidence, retrieves relevant repair principles from a general memory, and applies execution-anchored edits.<n>It substantially outperforms one-shot baselines, improving the base agent's success rate on SkillsBench from 36.05% to 61.63%.
  arXiv  Detail & Related papers  (2026-05-31T10:19:13Z)
• Skill-R1: Agent Skill Evolution via Reinforcement Learning [84.35984979949502]
  Skill-R1 is a reinforcement learning framework for instance-level recurrent skill optimization from verifiable rewards.<n>It preserves black-box compatibility with both open- and closed-source models while making adaptation substantially cheaper than model-level updates.
  arXiv  Detail & Related papers  (2026-05-10T06:19:15Z)
• SkillGen: Verified Inference-Time Agent Skill Synthesis [60.927977774369516]
  SkillGen is a framework that synthesizes a single auditable skill from trajectories generated by a base agent.<n>It identifies reusable success patterns, recurring failure modes, and behaviors that appear in nearby successes but are missing from failures.<n>A key novelty in SkillGen is that we model agent skills as interventions to empirically verify the net effect of skills on the overall performance.
  arXiv  Detail & Related papers  (2026-05-09T19:24:11Z)
• From Context to Skills: Can Language Models Learn from Context Skillfully? [56.5857437218136]
  Ctx2Skill is a self-evolving framework that autonomously discovers, refines, and selects context-specific skills.<n>It can be plugged into any language model to obtain better context learning capability.
  arXiv  Detail & Related papers  (2026-04-30T09:53:15Z)
• SkillX: Automatically Constructing Skill Knowledge Bases for Agents [27.56549031749266]
  SkillX is a fully automated framework for constructing a plug-and-play skill knowledge base.<n>It operates through a fully automated pipeline built on three synergistic innovations.<n> Experiments show that SkillKB consistently improves task success and execution efficiency when plugged into weaker base agents.
  arXiv  Detail & Related papers  (2026-04-06T16:09:33Z)
• SkillAttack: Automated Red Teaming of Agent Skills through Attack Path Refinement [66.44008181092832]
  LLM-based agent systems increasingly rely on agent skills sourced from open registries to extend their capabilities.<n>We introduce SkillAttack, a framework that verifies skill vulnerability exploitability through adversarial prompting.
  arXiv  Detail & Related papers  (2026-04-05T06:25:11Z)
• SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization [61.37478254738943]
  Inference-time skill augmentation is fundamentally limited.<n>We introduce SKILL0, an in-context reinforcement learning framework designed for skill internalization.<n>SKILL0 achieves substantial improvements over the standard RL baseline.
  arXiv  Detail & Related papers  (2026-04-02T17:03:05Z)
• EvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification [85.3692584167951]
  Anthropic proposes the concept of skills for LLM agents to tackle multi-step professional tasks.<n>A tool is a single, self-contained function, whereas a skill is a structured bundle of interdependent multi-file artifacts.<n>EvoSkills is a self-evolving skills framework that enables agents to autonomously construct complex, multi-file skill packages.
  arXiv  Detail & Related papers  (2026-04-02T06:43:20Z)
• EvoSkill: Automated Skill Discovery for Multi-Agent Systems [6.319876096746374]
  We introduce textbfEvoSkill, a self-evolving framework that automatically discovers and refines agent skills.<n>EvoSkill analyzes execution failures, proposes new skills or edits to existing ones, and materializes them into structured, reusable skill folders.<n>We evaluate EvoSkill on two benchmarks: OfficeQA, a grounded reasoning benchmark over U.S. Treasury data, and SealQA, a noisy retrieval benchmark.
  arXiv  Detail & Related papers  (2026-03-03T09:07:22Z)
• Uni-Skill: Building Self-Evolving Skill Repository for Generalizable Robotic Manipulation [32.86306309089796]
  Uni-Skill is a skill-centric framework that supports skill-aware planning.<n>Uni-Skill requests for new skill implementations when existing ones are insufficient.<n>Skill is a VerbNet-inspired repository derived from large-scale unstructured robotic videos.
  arXiv  Detail & Related papers  (2026-03-03T05:49:37Z)
• SkillNet: Create, Evaluate, and Connect AI Skills [159.47504178122156]
  SkillNet is an open infrastructure designed to create, evaluate, and organize AI skills at scale.<n>Our infrastructure integrates a repository of over 200,000 skills, an interactive platform, and a versatile Python toolkit.
  arXiv  Detail & Related papers  (2026-02-26T14:24:02Z)
• SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks [61.89812116484928]
  Agent Skills are structured packages of procedural knowledge that augment LLM agents at inference time.<n>We present SkillsBench, a benchmark of 86 tasks across 11 domains paired with curated Skills and deterministic verifiers.<n>We show curated Skills raise average pass rate by 16.2 percentage points(pp), but effects vary widely by domain.<n>Self-generated Skills provide no benefit on average, showing that models cannot reliably author the procedural knowledge they benefit from consuming.
  arXiv  Detail & Related papers  (2026-02-13T07:06:06Z)
• Discovering Generalizable Skills via Automated Generation of Diverse Tasks [82.16392072211337]
  We propose a method to discover generalizable skills via automated generation of a diverse set of tasks. As opposed to prior work on unsupervised discovery of skills, our method pairs each skill with a unique task produced by a trainable task generator. A task discriminator defined on the robot behaviors in the generated tasks is jointly trained to estimate the evidence lower bound of the diversity objective. The learned skills can then be composed in a hierarchical reinforcement learning algorithm to solve unseen target tasks.
  arXiv  Detail & Related papers  (2021-06-26T03:41:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.