The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents

• URL: http://arxiv.org/abs/2604.10577v2
• Date: Fri, 17 Apr 2026 07:20:03 GMT
• Title: The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents
• Abstract summary: Computer-use agents (CUAs) can now autonomously complete complex tasks in real environments, but when misled, they can also be automated.<n>We introduce OSBLIND, a benchmark that evaluates CUAs under unintended attack conditions.<n>Most CUAs exceed 90% attack success rate (ASR), and even safety-aligned Claude 4.5 Sonnet reaches 72.7% ASR.
• Score: 29.98056047452529
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Computer-use agents (CUAs) can now autonomously complete complex tasks in real digital environments, but when misled, they can also be used to automate harmful actions programmatically. Existing safety evaluations largely target explicit threats such as misuse and prompt injection, but overlook a subtle yet critical setting where user instructions are entirely benign and harm arises from the task context or execution outcome. We introduce OS-BLIND, a benchmark that evaluates CUAs under unintended attack conditions, comprising 300 human-crafted tasks across 12 categories, 8 applications, and 2 threat clusters: environment-embedded threats and agent-initiated harms. Our evaluation on frontier models and agentic frameworks reveals that most CUAs exceed 90% attack success rate (ASR), and even the safety-aligned Claude 4.5 Sonnet reaches 73.0% ASR. More interestingly, this vulnerability becomes even more severe, with ASR rising from 73.0% to 92.7% when Claude 4.5 Sonnet is deployed in multi-agent systems. Our analysis further shows that existing safety defenses provide limited protection when user instructions are benign. Safety alignment primarily activates within the first few steps and rarely re-engages during subsequent execution. In multi-agent systems, decomposed subtasks obscure the harmful intent from the model, causing safety-aligned models to fail. We will release our OS-BLIND to encourage the broader research community to further investigate and address these safety challenges.

Related papers

• SIR: Self-improving Red-teaming for Compute Use Agents [71.71987044117371]
  Computer use agents (CUAs) are vision-language models that perceive a screen and act on a real operating system through mouse, keyboard, and terminal.<n>Because they can be exposed to untrusted content while operating, they are vulnerable to indirect prompt injection (IPI)<n>We present SIR, a black box IPI attack that composes stealthy injections from a small library of reusable principles stated in plain language.
  arXiv  Detail & Related papers  (2026-08-31T03:39:43Z)
• Plant, Persist, Trigger: Sleeper Attack on Large Language Model Agents [64.80318459073526]
  We show that adversarial content can persist across interactions served by the same agent, making such threats harder to detect and mitigate.<n>Specifically, adversarial content may persist in the agent state, remain dormant across interactions, and later be activated by a benign user query.<n>We formalize this type of safety threat as Sleeper Attack. To evaluate it, we construct a benchmark with 1,896 instances covering six real-world harmful outcomes, three attack strategies, and three agent state targets.
  arXiv  Detail & Related papers  (2026-05-27T09:25:37Z)
• Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety [2.661610409070365]
  We introduce Boiling the Frog, a benchmark that evaluates whether tool-using AI models are susceptible to incremental attacks.<n> Scenarios are organized through a three-level operational risk taxonomy.<n>Across a nine-model panel, aggregate strict attack success rate (ASR) is 44.4%.
  arXiv  Detail & Related papers  (2026-05-21T15:50:18Z)
• ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks? [92.21756459993695]
  Exploitation is a challenging task because it requires low-level program reasoning.<n>Despite its importance and diagnostic value, exploitation remains under-evaluated.<n>We introduce ExploitGym, a large-scale, diverse, realistic benchmark on the exploitation capabilities of AI agents.
  arXiv  Detail & Related papers  (2026-05-11T18:00:14Z)
• Your Agent, Their Asset: A Real-World Safety Analysis of OpenClaw [87.97230960702274]
  We present the first real-world safety evaluation of OpenClaw.<n>We introduce the CIK taxonomy, which unifies an agent's persistent state into three dimensions.<n>Our evaluations cover 12 attack scenarios on a live OpenClaw instance.
  arXiv  Detail & Related papers  (2026-04-06T15:27:05Z)
• Trojan's Whisper: Stealthy Manipulation of OpenClaw through Injected Bootstrapped Guidance [23.059379933610163]
  Guidance injection is a stealthy attack vector that embeds adversarial operational narratives into bootstrap guidance files.<n>We construct 26 malicious skills spanning 13 attack categories including exfiltration, workspace destruction, privilege escalation, and persistent backdoor installation.<n>Our attacks achieve success rates from 16.4% to 64.2%, with the majority of malicious actions executed autonomously without user confirmation.
  arXiv  Detail & Related papers  (2026-03-20T14:17:56Z)
• Internal Safety Collapse in Frontier Large Language Models [65.00730294617382]
  This work identifies a critical failure mode in frontier large language models (LLMs)<n>Under certain task conditions, models enter a state in which they continuously generate harmful content while executing otherwise benign tasks.<n>We introduce TVD, a framework that triggers ISC through domain tasks where generating harmful content is the only valid completion.
  arXiv  Detail & Related papers  (2026-03-04T12:55:34Z)
• ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack [52.17935054046577]
  We present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks.<n>ReasAlign incorporates structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks.
  arXiv  Detail & Related papers  (2026-01-15T08:23:38Z)
• Securing AI Agents Against Prompt Injection Attacks [0.0]
  We present a benchmark for evaluating prompt injection risks in RAG-enabled AI agents.<n>Our framework reduces successful attack rates from 73.2% to 8.7% while maintaining 94.3% of baseline task performance.
  arXiv  Detail & Related papers  (2025-11-19T10:00:54Z)
• Jailbreak Mimicry: Automated Discovery of Narrative-Based Jailbreaks for Large Language Models [0.0]
  Large language models (LLMs) remain vulnerable to sophisticated prompt engineering attacks.<n>We introduce Jailbreak Mimicry, a systematic methodology for training compact attacker models to automatically generate narrative-based jailbreak prompts.<n>Our approach transforms adversarial prompt discovery from manual craftsmanship into a reproducible scientific process.
  arXiv  Detail & Related papers  (2025-10-24T23:53:16Z)
• STAC: When Innocent Tools Form Dangerous Chains to Jailbreak LLM Agents [38.755035623707656]
  This paper introduces Sequential Tool Attack Chaining (STAC), a novel multi-turn attack framework that exploits agent tool use.<n>We apply our framework to automatically generate and evaluate 483 STAC cases, featuring 1,352 sets of user-agent-environment interactions.<n>Our evaluations show that state-of-the-art LLM agents, including GPT-4.1, are highly vulnerable to STAC, with attack success rates (ASR) exceeding 90% in most cases.
  arXiv  Detail & Related papers  (2025-09-30T00:31:44Z)
• OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety [58.201189860217724]
  We introduce OpenAgentSafety, a comprehensive framework for evaluating agent behavior across eight critical risk categories.<n>Unlike prior work, our framework evaluates agents that interact with real tools, including web browsers, code execution environments, file systems, bash shells, and messaging platforms.<n>It combines rule-based analysis with LLM-as-judge assessments to detect both overt and subtle unsafe behaviors.
  arXiv  Detail & Related papers  (2025-07-08T16:18:54Z)
• OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents [60.78202583483591]
  We introduce OS-Harm, a new benchmark for measuring safety of computer use agents.<n> OS-Harm is built on top of the OSWorld environment and aims to test models across three categories of harm: deliberate user misuse, prompt injection attacks, and model misbehavior.<n>We evaluate computer use agents based on a range of frontier models and provide insights into their safety.
  arXiv  Detail & Related papers  (2025-06-17T17:59:31Z)
• AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions [64.85086226439954]
  We present SAFE, a benchmark for assessing the safety of embodied VLM agents on hazardous instructions.<n> SAFE comprises three components: SAFE-THOR, SAFE-VERSE, and SAFE-DIAGNOSE.<n>We uncover systematic failures in translating hazard recognition into safe planning and execution.
  arXiv  Detail & Related papers  (2025-06-17T16:37:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.