Black-Box Skill Stealing Attack from Proprietary LLM Agents: An Empirical Study

• URL: http://arxiv.org/abs/2604.21829v2
• Date: Mon, 27 Apr 2026 15:06:33 GMT
• Title: Black-Box Skill Stealing Attack from Proprietary LLM Agents: An Empirical Study
• Abstract summary: Large language model (LLM) agents rely on skills to package reusable capabilities through instructions, tools, and resources.<n>High-quality skills embed expert knowledge, curated, and execution constraints into agents.<n> adversaries can interact with public agent interfaces to extract hidden proprietary skill content.
• Score: 32.698841771877824
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) agents increasingly rely on skills to package reusable capabilities through instructions, tools, and resources. High-quality skills embed expert knowledge, curated workflows, and execution constraints into agents, fueling a growing skill economy through their value and scalability. Yet this ecosystem also creates a new attack surface, as adversaries can interact with public agent interfaces to extract hidden proprietary skill content. We present the first systematic study of black-box skill stealing against LLM agent systems. Compared with conventional system prompt stealing, skill stealing targets modular and structured capability packages whose leakage is directly actionable for copying, redistribution, and monetization, making the resulting harm potentially greater. To study this threat, we derive an attack taxonomy from prior prompt-stealing methods and build an automated stealing prompt generation agent. Starting from model-generated seed prompts, the framework expands attacks through scenario rationalization and structure injection while enforcing diversity via embedding-based filtering, yielding a reproducible pipeline for evaluating proprietary agent systems. We evaluate these attacks across commercial agent platforms and representative LLMs. Our results show that agent skills can often be extracted easily, posing a serious copyright risk. To mitigate this threat, we design defenses across the agent pipeline, focusing on input, inference, and output phase. Although these defenses substantially reduce leakage, the attack remains inexpensive and repeatable, and a single successful attempt is sufficient to compromise the protected skill. Overall, our findings suggest that these copyright risks remain largely overlooked across proprietary agent ecosystems, motivating stronger protection mechanisms.

Related papers

• AgentLeak: Cloning Stronger LLM Agent Capabilities onto Weaker Agents Beyond Skill Stealing [18.52239289354215]
  We show that a weaker agent may possess the same skills but still fail because it lacks procedural behaviors implicitly realized by the stronger agent.<n>Our key insight is that the skill execution gap itself forms a leakage surface, where missing behaviors are exposed.<n>We present AgentLeak, a black-box capability-cloning attack that identifies capability-critical behaviors from these execution differences and incorporates them into attacker-side skills.
  arXiv  Detail & Related papers  (2026-09-07T07:30:02Z)
• From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrails [9.514819678986488]
  LLM-based guardrails have emerged as a highly effective defense against prompt injection and jailbreak attacks in autonomous agents.<n>We show that attackers can inject crafted data to trap the guardrail in extended reasoning loops, effectuating a systematic denial-of-service attack.<n>We show that a single poisoned document can saturate shared guardrail infrastructures, effectively starving co-located agents and paralyzing the entire system.
  arXiv  Detail & Related papers  (2026-06-12T14:49:00Z)
• SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction [89.03272793385054]
  Agent skills occupy a privileged position in the agent workflow, rendering third-party skills a vulnerable attack surface.<n>We introduce SkillHarm, a benchmark of skill-based attacks across the skill-use lifecycle.<n>We show that current agents remain vulnerable with attack success rates up to 86.3% in FPP and 69.3% in SMP.
  arXiv  Detail & Related papers  (2026-06-01T17:45:39Z)
• Prompt-Unknown Promotion Attacks against LLM-based Sequential Recommender Systems [51.504307822017985]
  Large language model-powered sequential recommender systems (LLM-SRSs) have recently demonstrated remarkable performance.<n>This paper investigates the item promotion attack in LLM-SRSs under a more realistic setting where both the system prompt and victim model are unknown to the attacker.
  arXiv  Detail & Related papers  (2026-04-26T10:09:26Z)
• SkillAttack: Automated Red Teaming of Agent Skills through Attack Path Refinement [66.44008181092832]
  LLM-based agent systems increasingly rely on agent skills sourced from open registries to extend their capabilities.<n>We introduce SkillAttack, a framework that verifies skill vulnerability exploitability through adversarial prompting.
  arXiv  Detail & Related papers  (2026-04-05T06:25:11Z)
• Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs [65.6660735371212]
  We present textbftextscJustAsk, a framework that autonomously discovers effective extraction strategies through interaction alone.<n>It formulates extraction as an online exploration problem, using Upper Confidence Bound--based strategy selection and a hierarchical skill space spanning atomic probes and high-level orchestration.<n>Our results expose system prompts as a critical yet largely unprotected attack surface in modern agent systems.
  arXiv  Detail & Related papers  (2026-01-29T03:53:25Z)
• A Survey on Agentic Security: Applications, Threats and Defenses [6.83318476483428]
  The rapid shift from passive LLMs to autonomous LLM-agents marks a new paradigm in cybersecurity.<n>While these agents can act as powerful tools for both offensive and defensive operations, the very agentic context introduces a new class of inherent security risks.<n>We provide a comprehensive taxonomy of over 150 papers, explaining how agents are used, the vulnerabilities they possess, and the countermeasures designed to protect them.
  arXiv  Detail & Related papers  (2025-10-07T20:32:20Z)
• Adversarial Reinforcement Learning for Large Language Model Agent Safety [20.704989548285372]
  Large Language Model (LLM) agents can leverage tools like Google Search to complete complex tasks.<n>Current defense strategies rely on fine-tuning LLM agents on datasets of known attacks.<n>We propose Adversarial Reinforcement Learning for Agent Safety (ARLAS), a novel framework that leverages adversarial reinforcement learning (RL) by formulating the problem as a two-player zero-sum game.
  arXiv  Detail & Related papers  (2025-10-06T23:09:18Z)
• Towards Unifying Quantitative Security Benchmarking for Multi Agent Systems [0.0]
  Evolving AI systems increasingly deploy multi-agent architectures where autonomous agents collaborate, share information, and delegate tasks through developing protocols.<n>One such risk is a cascading risk: a breach in one agent can cascade through the system, compromising others by exploiting inter-agent trust.<n>In an ACI attack, a malicious input or tool exploit injected at one agent leads to cascading compromises and amplified downstream effects across agents that trust its outputs.
  arXiv  Detail & Related papers  (2025-07-23T13:51:28Z)
• AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents [54.29555239363013]
  We propose a generic black-box fuzzing framework, AgentVigil, to automatically discover and exploit indirect prompt injection vulnerabilities.<n>We evaluate AgentVigil on two public benchmarks, AgentDojo and VWA-adv, where it achieves 71% and 70% success rates against agents based on o3-mini and GPT-4o.<n>We apply our attacks in real-world environments, successfully misleading agents to navigate to arbitrary URLs, including malicious sites.
  arXiv  Detail & Related papers  (2025-05-09T07:40:17Z)
• Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs [83.11815479874447]
  We propose a novel jailbreak attack framework, inspired by cognitive decomposition and biases in human cognition.<n>We employ cognitive decomposition to reduce the complexity of malicious prompts and relevance bias to reorganize prompts.<n>We also introduce a ranking-based harmfulness evaluation metric that surpasses the traditional binary success-or-failure paradigm.
  arXiv  Detail & Related papers  (2025-05-03T05:28:11Z)
• Real AI Agents with Fake Memories: Fatal Context Manipulation Attacks on Web3 Agents [36.49717045080722]
  This paper investigates the vulnerabilities of AI agents within blockchain-based financial ecosystems when exposed to adversarial threats in real-world scenarios.<n>We introduce the concept of context manipulation -- a comprehensive attack vector that exploits unprotected context surfaces.<n>Using ElizaOS, we showcase that malicious injections into prompts or historical records can trigger unauthorized asset transfers and protocol violations.
  arXiv  Detail & Related papers  (2025-03-20T15:44:31Z)
• Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks [88.84977282952602]
  A high volume of recent ML security literature focuses on attacks against aligned large language models (LLMs)<n>In this paper, we analyze security and privacy vulnerabilities that are unique to LLM agents.<n>We conduct a series of illustrative attacks on popular open-source and commercial agents, demonstrating the immediate practical implications of their vulnerabilities.
  arXiv  Detail & Related papers  (2025-02-12T17:19:36Z)
• Targeting the Core: A Simple and Effective Method to Attack RAG-based Agents via Direct LLM Manipulation [4.241100280846233]
  AI agents, powered by large language models (LLMs), have transformed human-computer interactions by enabling seamless, natural, and context-aware communication.<n>This paper investigates a critical vulnerability: adversarial attacks targeting the LLM core within AI agents.
  arXiv  Detail & Related papers  (2024-12-05T18:38:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.