Agent Skill Security: Threat Models, Attacks, Defenses, and Evaluation

• URL: http://arxiv.org/abs/2607.13987v1
• Date: Wed, 15 Jul 2026 16:15:46 GMT
• Title: Agent Skill Security: Threat Models, Attacks, Defenses, and Evaluation
• Abstract summary: We present SkillSec-Eval, a lifecycle-aware framework for evaluating the security of reusable agent skills.<n>We first characterize the skill lifecycle and develop a threat taxonomy spanning repository admission, semantic retrieval, planner selection, execution, and skill evolution.<n>We then instantiate this taxonomy in SkillSec-Eval and conduct a comprehensive empirical evaluation using a repository of 327 real-world skills.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reusable skills are becoming a fundamental building block of Large Language Model (LLM) agents, enabling capabilities to be packaged, shared, and reused across diverse applications. However, existing security research primarily focuses on prompt injection and runtime execution, leaving security risks throughout the broader skill lifecycle largely unexplored. In this paper, we present SkillSec-Eval, a lifecycle-aware framework for systematically evaluating the security of reusable agent skills. We first characterize the skill lifecycle and develop a threat taxonomy spanning repository admission, semantic retrieval, planner selection, execution, and skill evolution. We then instantiate this taxonomy in SkillSec-Eval and conduct a comprehensive empirical evaluation using a repository of 327 real-world skills. Our study demonstrates that vulnerabilities arise at multiple lifecycle stages beyond execution, highlighting the need for lifecycle-aware security analysis of reusable agent skills.

Related papers

• CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wild [49.44069582581181]
  CyberFactory transforms public vulnerability artifacts, including CVEs from the wild, into executable and verifiable task instances.<n>It uses a reusable vulnerability-analysis skill to guide the teacher through source inspection, problem solving with domain prior, and evidence-based validation.<n>On CyberGym, modelname reaches 52.4% Pass@1 under a one-hour budget, improving over its Qwen3.5 base model by +22.8 points.
  arXiv  Detail & Related papers  (2026-08-24T12:30:10Z)
• Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents [67.97365535829098]
  Learning from experience is critical for developing capable, self-improving large language model (LLM) agents.<n>We propose Evo-Harness, which distills noisy, single-shot executions into reusable skill harnesses for cross-domain and topic-level adaptation.<n>Our analysis demonstrates the effectiveness of Evo-Harness and provides a principled understanding of how LLM agents can effectively learn on the fly.
  arXiv  Detail & Related papers  (2026-08-15T06:43:56Z)
• Inside the Skill Market: From Software Engineering Activities to Reusable Agent Skills [17.375016703429164]
  Software engineering (abbrev. SE) has continuously evolved through increasingly powerful forms of reuse.<n>Recent advances in AI agents have introduced a potentially new reusable artifact: skills.<n>Agent skill repositories and marketplaces enable developers to package, share, and reuse SE expertise as reusable skills.
  arXiv  Detail & Related papers  (2026-07-10T03:21:16Z)
• Yuvion LLM: An Adversarially-Aware Large Language Model for Content And AI Safety [67.86055044301408]
  We present Yuvion LLM, a large language model built for adversarially robust content safety and broader AI safety.<n>Its pipeline combines adversarially aware data construction, knowledge-enhanced continued pretraining, and policy-grounded multi-task safety post-training.<n>Yuvion LLM demonstrates clear advantages on safety-focused benchmarks and particularly strong robustness under adversarial conditions.
  arXiv  Detail & Related papers  (2026-06-26T01:12:02Z)
• SkillAudit: From Fixed-Suite Benchmarking to Skill-Centered Assessment [10.850946672552277]
  We introduce SkillAudit, an end-to-end framework for skill-centered assessment.<n>It generates a comprehensive, multi-dimensional evaluation report spanning utility, efficiency/cost, and safety.<n>After scanning top-ranked real-world skill packages, we found that over 7% of skills are at risky status.
  arXiv  Detail & Related papers  (2026-06-21T17:40:05Z)
• SeClaw: Spec-Driven Security Task Synthesis for Evaluating Autonomous Agents [87.26967184869198]
  SeClaw is a framework that combines specification-driven security task synthesis with execution-based security evaluation for Autonomous agents.<n> benchmark covers risks arising from resources, user tasks, environments, and intrinsic agent behaviors.
  arXiv  Detail & Related papers  (2026-06-01T14:23:42Z)
• A Systematic Security Evaluation of OpenClaw and Its Variants [26.64519805689193]
  This paper presents a systematic security assessment of six representative OpenClaw-series agent frameworks.<n>We construct a benchmark of 205 test cases covering representative attack behaviors across the full agent execution lifecycle.<n>Our results show that all evaluated agents exhibit substantial security vulnerabilities, and that agentized systems are significantly riskier than their underlying models.
  arXiv  Detail & Related papers  (2026-04-03T15:52:36Z)
• SoK: Agentic Skills -- Beyond Tool Use in LLM Agents [6.356997609995175]
  Agentic systems increasingly rely on reusable procedural capabilities, textita.k.a., agentic skills, to execute long-horizon reliably.<n>This paper maps the skill layer across the full lifecycle (discovery, practice, distillation, storage, composition, evaluation, and update)<n>We analyze the security and governance implications of skill-based agents, covering supply-chain risks, prompt injection via skill payloads, and trust-tiered execution.
  arXiv  Detail & Related papers  (2026-02-24T13:11:38Z)
• Agent Skills: A Data-Driven Analysis of Claude Skills for Extending Large Language Model Functionality [9.192260493061754]
  Agent skills extend large language model (LLM) agents with reusable, program-like modules.<n>We conduct a large-scale, data-driven analysis of 40,285 publicly listed skills from a major marketplace.<n>Our results show that skill publication tends to occur in short bursts that track shifts in community attention.
  arXiv  Detail & Related papers  (2026-02-08T15:14:12Z)
• Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety [296.5392512998251]
  We present a comprehensive taxonomy of safety threats to large models, including adversarial attacks, data poisoning, backdoor attacks, jailbreak and prompt injection attacks, energy-latency attacks, data and model extraction attacks, and emerging agent-specific threats.<n>We identify and discuss the open challenges in large model safety, emphasizing the need for comprehensive safety evaluations, scalable and effective defense mechanisms, and sustainable data practices.
  arXiv  Detail & Related papers  (2025-02-02T05:14:22Z)
• EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [53.717918131568936]
  Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction.<n>Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results.<n>However, the deployment of these agents in physical environments presents significant safety challenges.<n>This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios.
  arXiv  Detail & Related papers  (2024-08-08T13:19:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.