AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skills

• URL: http://arxiv.org/abs/2605.13940v1
• Date: Wed, 13 May 2026 17:04:17 GMT
• Title: AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skills
• Abstract summary: Third-party skills are becoming the package ecosystem for LLM agents.<n>Malicious skills do not need to ask the model to perform an obviously harmful action.<n>Instead, it can disguise the harmful behavior as part of a routine workflow.<n>We introduce AgentTrap, a benchmark for evaluating whether LLM agents can use third-party skills while resisting malicious runtime behavior.
• Score: 32.24433022603674
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Third-party skills are becoming the package ecosystem for LLM agents. They package natural-language instructions, helper scripts, templates, documents, and service configuration into reusable workflows. This makes skills useful, but it also introduces a new security problem: a malicious skill does not need to ask the model to perform an obviously harmful action. Instead, it can disguise the harmful behavior as part of a routine workflow, relying on the agent to execute that workflow with high-value permissions and limited human supervision. We introduce AgentTrap, a dynamic benchmark for evaluating whether LLM agents can use third-party skills while resisting malicious runtime behavior. AgentTrap contains 141 tasks: 91 malicious tasks and 50 benign utility tasks, covering 16 security-impact dimensions grounded in agent-skill supply-chain threats. In each task, the agent receives an ordinary user request, runs with installed skills that may contain malicious workflow elements, and is executed in a sandboxed environment. AgentTrap then judges complete trajectories for attack success, blocked or refused behavior, attack-not-triggered cases, and no-attack-evidence outcomes. Our central finding is that the most informative failures are not simple jailbreaks. Models often complete the visible user task while treating unsafe side effects introduced by the skill as part of the normal workflow. This motivates runtime evaluation of the concrete model--framework--workspace environment in which users actually delegate work. Code and data are available at https://github.com/zhmzm/AgentTrap and https://huggingface.co/datasets/zhmzm/AgentTrap.

Related papers

• Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity? [54.701199583505144]
  Cross-application access creates a privacy risk that has been largely overlooked.<n>We introduce AgentCIBench, an evaluation harness that turns this risk into executable, deterministically scored scenarios.<n>We evaluate 15 frontier agents and find a surprisingly high failure rate.
  arXiv  Detail & Related papers  (2026-06-22T11:36:58Z)
• TRACE: Task-Aware Adaptive Self-Evolving Agentic Jailbreaking [40.259937753172224]
  The rise of LLM agents introduces a new threat by enabling planning, coding, and even end-to-end execution of expert-level attack.<n>In this paper, we propose TRACE, a practical agentic jailbreaking framework to further reveal the risks of this threat surface.
  arXiv  Detail & Related papers  (2026-05-29T06:13:58Z)
• AgentHijack: Benchmarking Computer Use Agent Robustness to Common Environment Corruptions [78.49000936275773]
  We introduce AgentHijack, a benchmark designed to evaluate the robustness of computer-use agents under common corruptions.<n>We evaluate a variety of desktop tasks that utilize MLLM-based agents and discover that even minor instances of corruption can result in substantial performance degradation.<n>We propose AgentHijack-Agent, a framework that integrates an action generator with enhanced grounding capabilities and an onlooker responsible for behavior summarization and environment checking.
  arXiv  Detail & Related papers  (2026-05-25T11:09:22Z)
• ClawSafety: "Safe" LLMs, Unsafe Agents [25.729388843970014]
  Personal AI agents like OpenClaw run with elevated privileges on users' local machines.<n>ClawSAFETY is a benchmark of 120 adversarial test scenarios organized along three dimensions (harm domain, attack vector, and harmful action type)<n>Each test case embeds adversarial content in one of three channels the agent encounters during normal work: workspace skill files, emails from trusted senders, and web pages.
  arXiv  Detail & Related papers  (2026-04-01T22:24:24Z)
• AgentWall: A Runtime Safety Layer for Local AI Agents [0.0]
  AgentWall is a runtime safety and observability layer for local AI agents.<n>It intercepts every proposed agent action before it reaches the host environment, evaluates it against an explicit declarative policy, requires human approval for sensitive operations, and records a complete execution trail for audit and replay.<n>We present the design, architecture, threat model, and policy model of AgentWall, and demonstrate 92.9% policy enforcement accuracy with sub-millisecond overhead across 14 benchmark tests.
  arXiv  Detail & Related papers  (2026-03-24T11:39:35Z)
• Tracking Capabilities for Safer Agents [2.9897366166831265]
  Instead of calling tools directly, agents express their intentions as code in a capability-safe language: Scala 3 with capture checking.<n> Scala's type system tracks capabilities statically, providing fine-grained control over what an agent can do.<n>Our experiments show that agents can generate capability-safe code with no significant loss in task performance.
  arXiv  Detail & Related papers  (2026-03-01T08:39:37Z)
• AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification [25.817251923574286]
  We propose a novel inference-time detection and mitigation framework for large language model (LLM) agents.<n>AgentSentry is the first inference-time defense to model multi-turn IPI as a temporal causal takeover.<n>We evaluate AgentSentry on the textscAgentDojo benchmark across four task suites, three IPI attack families, and multiple black-box LLMs.
  arXiv  Detail & Related papers  (2026-02-26T07:59:10Z)
• SkillJect: Automating Stealthy Skill-Based Prompt Injection for Coding Agents with Trace-Driven Closed-Loop Refinement [120.52289344734415]
  We propose an automated framework for stealthy prompt injection tailored to agent skills.<n>The framework forms a closed loop with three agents: an Attack Agent that synthesizes injection skills under explicit stealth constraints, a Code Agent that executes tasks using the injected skills and an Evaluate Agent that logs action traces.<n>Our method consistently achieves high attack success rates under realistic settings.
  arXiv  Detail & Related papers  (2026-02-15T16:09:48Z)
• Defeating Prompt Injections by Design [79.00910871948787]
  CaMeL is a robust defense that creates a protective system layer around the Large Language Models.<n>To operate, CaMeL explicitly extracts the control and data flows from the (trusted) query.<n>To further improve security, CaMeL uses a notion of a capability to prevent the exfiltration of private data over unauthorized data flows.
  arXiv  Detail & Related papers  (2025-03-24T15:54:10Z)
• AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents [84.96249955105777]
  LLM agents may pose a greater risk if misused, but their robustness remains underexplored.<n>We propose a new benchmark called AgentHarm to facilitate research on LLM agent misuse.<n>We find leading LLMs are surprisingly compliant with malicious agent requests without jailbreaking.
  arXiv  Detail & Related papers  (2024-10-11T17:39:22Z)
• Dissecting Adversarial Robustness of Multimodal LM Agents [70.2077308846307]
  We manually create 200 targeted adversarial tasks and evaluation scripts in a realistic threat model on top of VisualWebArena.<n>We find that we can successfully break latest agents that use black-box frontier LMs, including those that perform reflection and tree search.<n>We also use ARE to rigorously evaluate how the robustness changes as new components are added.
  arXiv  Detail & Related papers  (2024-06-18T17:32:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.