One Goal, Many Commands: Characterizing Denylist Fragility in AI Agents

• URL: http://arxiv.org/abs/2606.15549v2
• Date: Sat, 20 Jun 2026 22:26:27 GMT
• Title: One Goal, Many Commands: Characterizing Denylist Fragility in AI Agents
• Abstract summary: Terminal AI agents rely heavily on shell command execution to interact with the host systems.<n>A built-in denylist of Claude Code, well-maintained by its developers, can overlook bypass commands that invalidate its effectiveness.<n>This paper presents the first systematic characterization of command denylist fragility in terminal AI agents.
• Score: 8.5338788749266
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The adoption of AI agents is increasing rapidly. Terminal AI agents, i.e., AI agents that run in terminal environments, are a widely used type of AI agents. Terminal AI agents rely heavily on shell command execution to interact with the host systems. They adopt a three-list command-gating mechanism to mitigate security risks introduced by command execution, with denylists serving as the load-bearing component. However, modern operating systems often ship a large, ever-expanding set of shell commands with complex functionalities. Our observation is that even a built-in denylist of Claude Code, well-maintained by its developers, can overlook bypass commands that invalidate its effectiveness. Such negligence leads to fragile command denylists that cannot even block operations that practitioners expect them to block. This paper presents the first systematic characterization of command denylist fragility in terminal AI agents. The paper formalizes the command denylist fragility problem and proposes an LLM-driven pipeline, ShellSieve, to detect such fragility. It prompts the LLM to propose possible bypasses and iteratively repairs them using feedback from a validator that executes them in a sandbox. In the evaluation, we applied ShellSieve to 1,709 real-world command denylists (containing 13,332 denylist rules) collected from GitHub. The evaluation shows several key findings, including that 69.0--98.6% of the denylists are fragile, that this fragility occurs consistently across projects and agents, and the validity of several possible root causes for this fragility. Our pipeline and findings will hopefully facilitate future research and practice regarding the command denylists used by AI agents.

Related papers

• Towards a Risk Assessment of Malicious Skill Files in Coding Agents [9.553517006800913]
  We make three contributions to the field of autonomous coding agents.<n>First, an adversarial skill-synthesis method to transform 471 real-world shell commands into benign-appearing skills.<n>Second, a reproducible evaluation pipeline coupling run stratification, evidence anchoring, a refusal veto, and a deterministic declared-intent override.<n>Third, a large-scale characterization of two enterprise-grade agents across 5,629 completed runs.
  arXiv  Detail & Related papers  (2026-08-05T11:33:43Z)
• MOSAIC: Knowledge-Guided CLI Command Composition Attack in LLM Coding Agents [12.605238431656927]
  MOSAIC is a knowledge-guided framework that distills validated command-state behaviors from CVEs, advisories, and researcher PoCs into reusable summaries.<n>We show that MOSAIC achieves a 96.59% attack success rate under benign developer tasks.
  arXiv  Detail & Related papers  (2026-07-03T01:41:17Z)
• Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens [65.53494487819053]
  Claw-like AI agents (e.g., OpenClaw) are always-on processes with persistent access to credentials, files, tools, and external services.<n>We adopt a computer-system analogy: treating a Claw-like agent as an agentic computer system whose gateway runtime plays an OS-like mediation role.<n>We develop SafeClawArena, a benchmark of 406 adversarial tasks across four attack surfaces.
  arXiv  Detail & Related papers  (2026-06-29T18:00:45Z)
• Securing LLM Agents Need Intent-to-Execution Integrity [49.490963596514185]
  We argue that securing LLM agents requires defining an end-to-end correctness property that specifies when an agent's execution faithfully reflects the user's intent.<n>We observe that LLM agents are structurally analogous to compilers, where security violations correspond to mis-executions that do not preserve user intent.<n>We derive four integrity properties that must hold simultaneously: emphTool Integrity, emphInstruction Integrity, emphJudgment Integrity, and emphData Flow Integrity.
  arXiv  Detail & Related papers  (2026-05-16T12:53:31Z)
• A Systematic Taxonomy of Security Vulnerabilities in the OpenClaw AI Agent Framework [9.723337441030283]
  Vulnerabilities cluster along two axes: (1) the system axis, reflecting the architectural layer (exec policy, gateway, channel, sandbox, browser, plugin, agent/prompt); and (2) the attack axis, reflecting adversarial techniques (identity spoofing, policy bypass, cross-layer composition, prompt injection, supply-chain escalation).<n>We present a systematic taxonomy of 190 advisories filed against OpenClaw, an open-source AI agent runtime, organized by architectural layer and trust-violation type.
  arXiv  Detail & Related papers  (2026-03-29T04:51:27Z)
• AgentWall: A Runtime Safety Layer for Local AI Agents [0.0]
  AgentWall is a runtime safety and observability layer for local AI agents.<n>It intercepts every proposed agent action before it reaches the host environment, evaluates it against an explicit declarative policy, requires human approval for sensitive operations, and records a complete execution trail for audit and replay.<n>We present the design, architecture, threat model, and policy model of AgentWall, and demonstrate 92.9% policy enforcement accuracy with sub-millisecond overhead across 14 benchmark tests.
  arXiv  Detail & Related papers  (2026-03-24T11:39:35Z)
• Agent-Sentry: Bounding LLM Agents via Execution Provenance [4.306721924522171]
  Agentic computing systems, which autonomously spawn new functionalities based on natural language instructions, are becoming increasingly prevalent.<n>While immensely capable, these systems raise serious security, privacy, and safety concerns.<n>We propose Agent-Sentry, a framework that attempts to bound agentic systems to address this problem.
  arXiv  Detail & Related papers  (2026-03-24T07:12:53Z)
• SkillJect: Automating Stealthy Skill-Based Prompt Injection for Coding Agents with Trace-Driven Closed-Loop Refinement [120.52289344734415]
  We propose an automated framework for stealthy prompt injection tailored to agent skills.<n>The framework forms a closed loop with three agents: an Attack Agent that synthesizes injection skills under explicit stealth constraints, a Code Agent that executes tasks using the injected skills and an Evaluate Agent that logs action traces.<n>Our method consistently achieves high attack success rates under realistic settings.
  arXiv  Detail & Related papers  (2026-02-15T16:09:48Z)
• The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search [58.8834056209347]
  Large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safety guardrails to elicit harmful outputs.<n>We introduce the Correlated Knowledge Attack Agent (CKA-Agent), a dynamic framework that reframes jailbreaking as an adaptive, tree-structured exploration of the target model's knowledge base.
  arXiv  Detail & Related papers  (2025-12-01T07:05:23Z)
• Agent Skills Enable a New Class of Realistic and Trivially Simple Prompt Injections [24.46526203453932]
  A frontier LLM company made a step towards this by introducing Agent Skills.<n>We show that they are fundamentally insecure, since they enable trivially simple prompt injections.<n>We demonstrate how to hide malicious instructions in long Agent Skill files and referenced scripts to exfiltrate sensitive data.
  arXiv  Detail & Related papers  (2025-10-30T10:27:11Z)
• "Your AI, My Shell": Demystifying Prompt Injection Attacks on Agentic AI Coding Editors [18.075392981522565]
  We show how attackers can remotely exploit these systems by poisoning external development resources with malicious instructions.<n>We implement AIShellJack, an automated testing framework for assessing prompt injection vulnerabilities in agentic AI coding editors.<n>Our evaluation results show that attack success rates can reach as high as 84% for executing malicious commands.
  arXiv  Detail & Related papers  (2025-09-26T08:20:54Z)
• OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents [60.78202583483591]
  We introduce OS-Harm, a new benchmark for measuring safety of computer use agents.<n> OS-Harm is built on top of the OSWorld environment and aims to test models across three categories of harm: deliberate user misuse, prompt injection attacks, and model misbehavior.<n>We evaluate computer use agents based on a range of frontier models and provide insights into their safety.
  arXiv  Detail & Related papers  (2025-06-17T17:59:31Z)
• AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents [54.29555239363013]
  We propose a generic black-box fuzzing framework, AgentVigil, to automatically discover and exploit indirect prompt injection vulnerabilities.<n>We evaluate AgentVigil on two public benchmarks, AgentDojo and VWA-adv, where it achieves 71% and 70% success rates against agents based on o3-mini and GPT-4o.<n>We apply our attacks in real-world environments, successfully misleading agents to navigate to arbitrary URLs, including malicious sites.
  arXiv  Detail & Related papers  (2025-05-09T07:40:17Z)
• AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents [84.96249955105777]
  LLM agents may pose a greater risk if misused, but their robustness remains underexplored.<n>We propose a new benchmark called AgentHarm to facilitate research on LLM agent misuse.<n>We find leading LLMs are surprisingly compliant with malicious agent requests without jailbreaking.
  arXiv  Detail & Related papers  (2024-10-11T17:39:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.