Defense-as-Skill: Evolving Runtime Guard Skill for Skill-Augmented Agents

• URL: http://arxiv.org/abs/2609.01487v1
• Date: Tue, 01 Sep 2026 16:19:31 GMT
• Title: Defense-as-Skill: Evolving Runtime Guard Skill for Skill-Augmented Agents
• Abstract summary: We propose Defense-as-Skill, a defense paradigm that implements the runtime guard itself as an installable, inspectable, and editable skill.<n>Our guard, SkillSonar, checks sensitive actions against the user's task boundary, routing each action to an allow, replan, or confirmation decision.
• Score: 30.77975669386977
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Skill-augmented agents load reusable skills as persistent runtime context, improving task performance but also giving malicious skills a durable channel for steering future actions. Such skills may leak secrets, corrupt code, bypass approvals, or stage data for exfiltration only after a concrete user task and workspace state make the unsafe action appear useful. This makes pre-install vetting insufficient and calls for runtime, task-conditioned protection. We propose Defense-as-Skill, a defense paradigm that implements the runtime guard itself as an installable, inspectable, and editable skill. Our guard, SkillSonar, runs alongside untrusted task skills and checks sensitive actions against the user's task boundary, routing each action to an allow, replan, or confirmation decision without modifying the underlying agent runtime. To study this setting, we construct SCOPE-R, a task-conditioned dataset covering 6 risk families and 21 sub-categories, with 206 attack-confirmed malicious instances and 43 benign tasks. We then improve SkillSonar on the SCOPE-R training subset using runtime guard-skill evolution, a Monte-Carlo Tree Search procedure that evolves the on-disk guard skill from feedback on the rollouts. Across Claude Code and OpenClaw, the evolved guard substantially reduces attack success while maintaining a favorable safety-utility trade-off. On repeated GLM-5 runs, SkillSonar reduces ID ASR from 0.482 to 0.104 and OOD ASR from 0.606 to 0.115. Further analyses demonstrate transfer across victim models, held-out risk families, and external benchmarks, as well as retained protection against adaptive attackers. Ablations further show that explicit safety responsibility assignment and the skill-native representation are both important to the observed gains.

Related papers

• CoER: Defending against Adaptive Indirect Prompt Injection via Adversarial Co-Evolution and Refinement [19.128257554975104]
  Language-model agents are vulnerable to indirect prompt injection (IPI) during tool use.<n>We propose CoER, a verifier-grounded co-evolution and refinement framework.<n>In our main seven-domain evaluation, CoER reduces observed overall attack success from 38.5% to 0.2%.
  arXiv  Detail & Related papers  (2026-09-07T14:09:11Z)
• TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation [13.298401993417611]
  TRUSS is an evidence guided framework for generating functionally effective and safety reliable Agent Skills.<n>We evaluate TRUSS on 168 SkillInject artifacts, 155 SkillSafetyBench cases, and all 187 tasks in SkillGenBench.<n>For Skill generation, TRUSS raises task effectiveness from 17.11% without Skills to 52.94%, while increasing the benchmark Security rate from 50.80% to 100.00%.
  arXiv  Detail & Related papers  (2026-08-18T09:52:57Z)
• Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents [7.8654950055159345]
  We introduce SkillMisevo-Gym, a lifecycle-aware harness that versions skill state across agent frameworks.<n>We also introduce SafeEvolve, a wrapper that repairs unsafe content and governs subsequent reuse.
  arXiv  Detail & Related papers  (2026-08-13T05:47:43Z)
• SkillGuard: A Permission Framework for Agent Skills [14.787581333168859]
  We present SkillGuard, a skill-centric permission framework that treats skills as permission-bearing executable artifacts.<n>We evaluate SkillGuard on 315 real-world skills and SkillInject.
  arXiv  Detail & Related papers  (2026-06-02T02:01:53Z)
• SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction [89.03272793385054]
  Agent skills occupy a privileged position in the agent workflow, rendering third-party skills a vulnerable attack surface.<n>We introduce SkillHarm, a benchmark of skill-based attacks across the skill-use lifecycle.<n>We show that current agents remain vulnerable with attack success rates up to 86.3% in FPP and 69.3% in SMP.
  arXiv  Detail & Related papers  (2026-06-01T17:45:39Z)
• On-Policy Self-Evolution via Failure Trajectories for Agentic Safety Alignment [54.30690671490447]
  Existing safety-alignment signals are largely response-level or off-policy.<n>FATE transforms verifier-scored failures into repair supervision without expert demonstrations.<n>FATE reduces attack success rate by 33.5%, harmful compliance by 82.6%, and improves external trajectory-safety diagnosis by 6.5%.
  arXiv  Detail & Related papers  (2026-05-12T09:56:28Z)
• SkillAttack: Automated Red Teaming of Agent Skills through Attack Path Refinement [66.44008181092832]
  LLM-based agent systems increasingly rely on agent skills sourced from open registries to extend their capabilities.<n>We introduce SkillAttack, a framework that verifies skill vulnerability exploitability through adversarial prompting.
  arXiv  Detail & Related papers  (2026-04-05T06:25:11Z)
• RoboSafe: Safeguarding Embodied Agents via Executable Safety Logic [56.38397499463889]
  Embodied agents powered by vision-language models (VLMs) are increasingly capable of executing complex real-world tasks.<n>However, they remain vulnerable to hazardous instructions that may trigger unsafe behaviors.<n>We propose RoboSafe, a runtime safeguard for embodied agents through executable predicate-based safety logic.
  arXiv  Detail & Related papers  (2025-12-24T15:01:26Z)
• AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning [78.5751183537704]
  AdvEvo-MARL is a co-evolutionary multi-agent reinforcement learning framework that internalizes safety into task agents.<n>Rather than relying on external guards, AdvEvo-MARL jointly optimize attackers and defenders.
  arXiv  Detail & Related papers  (2025-10-02T02:06:30Z)
• Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models [64.47869632167284]
  Conventional language model (LM) safety alignment relies on a reactive, disjoint procedure: attackers exploit a static model, followed by defensive fine-tuning to patch exposed vulnerabilities.<n>This sequential approach creates a mismatch -- attackers overfit to obsolete defenses, while defenders perpetually lag behind emerging threats.<n>We propose Self-RedTeam, an online self-play reinforcement learning algorithm where an attacker and defender agent co-evolve through continuous interaction.
  arXiv  Detail & Related papers  (2025-06-09T06:35:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.