Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection

• URL: http://arxiv.org/abs/2609.04533v1
• Date: Thu, 03 Sep 2026 22:44:45 GMT
• Title: Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection
• Abstract summary: We present Repeat-After-Me, a black-box adaptive visual prompt injection attack that can reveal personally identifiable information or make malicious tool calls.<n>In our evaluation, injections optimized on one surrogate retain 43-46% of the original ASR on two commercial victims.<n>In a default OpenClaw Discord deployment, an untrusted user can use a minimally injected image to overwrite TOOLS.md, enabling future sensitive behaviors like remote code execution and secret exfiltration.
• Score: 40.37588773528427
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prompt injection is widely recognized as a major security threat to AI agents that interact with untrusted external data, such as websites, documents, and emails. Prior work has shown that, in the text domain, black-box prompt injection can achieve near-perfect attack success rates (ASRs). In the image domain, however, existing visual prompt injection methods are substantially less effective in attacking frontier commercial VLMs for materially harmful behavior. Achieving such outputs is hard because it requires a long and/or format-compliant target string, such as a precise, parseable native tool call with exact function names and arguments. We present Repeat-After-Me, a black-box adaptive visual prompt injection attack that can reveal personally identifiable information or make malicious tool calls. Across both open-weight and commercial frontier VLMs, including Qwen3.6-27B and GPT-5.5, our method achieves ASRs exceeding 80% and 47%, respectively, under a realistic setting in which the benign user prompt is semantically unrelated to the injected task and does not verbally authorize it. In our evaluation, injections optimized on one surrogate retain 43-46% of the original ASR on two commercial victims, and cross-sample transferability retains 64-66% of the original ASR on those two models. We test our attack in a real-world OpenClaw agent: in a default OpenClaw Discord deployment, an untrusted user can use a minimally injected image to overwrite TOOLS.md, enabling future sensitive behaviors like remote code execution and secret exfiltration. We show our new attack vector works in cases where adaptive textual prompt injection fails. We discuss potential defenses.

Related papers

• SIR: Self-improving Red-teaming for Compute Use Agents [71.71987044117371]
  Computer use agents (CUAs) are vision-language models that perceive a screen and act on a real operating system through mouse, keyboard, and terminal.<n>Because they can be exposed to untrusted content while operating, they are vulnerable to indirect prompt injection (IPI)<n>We present SIR, a black box IPI attack that composes stealthy injections from a small library of reusable principles stated in plain language.
  arXiv  Detail & Related papers  (2026-08-31T03:39:43Z)
• ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection [61.4148196085256]
  We present ROPE (Routed Origin Policy Enforcement), which is anchored in a structural notion of trust.<n>Our approach admits two provable guarantees: 1) at every step of a trajectory, no value whose only origin is attacker-writable content reaches an origin-guarded parameter, and 2) no rewording of an injection changes an admission decision.
  arXiv  Detail & Related papers  (2026-08-27T01:22:14Z)
• SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation [11.545542144576395]
  Prompt injections are listed as the #1 threat to AI agents.<n>We propose Secure On-Policy Distillation (SecOPD) that provides token-level feedback to guide fine-tuning.<n>SecOPD achieves a 9.0% ASR against the SoTA PISmith adaptive prompt injections, compared to 94.0% for the prior SoTA, Meta-SecAlign.
  arXiv  Detail & Related papers  (2026-08-21T16:14:07Z)
• MemVenom: Triggered Poisoning of Multimodal Memories in Web Agents [74.64265314956441]
  We propose a unified black-box attack framework that poisons graph-structured external memory with coordinated text-image evidence.<n>MemVenom achieves strong end-to-end attack success with minimal impact on benign performance, reaching up to 99.15% on GPT-5-family web agents.
  arXiv  Detail & Related papers  (2026-06-09T11:53:25Z)
• An Empirical Study of Privacy Leakage Chains via Prompt Injection in Black-Box Chatbot Environments [3.275342475988451]
  This paper studies a privacy-leakage attack chain based on indirect prompt injection.<n>We first analyze how an attacker can hijack an agent's intended task by crafting external content that appears benign to the victim.<n>We then evaluate a new prompt-language technique, called exemplification, which uses a bridge in the external content to reframe the user prompt and the benign beginning of the retrieved page.
  arXiv  Detail & Related papers  (2026-05-18T09:38:18Z)
• Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection [37.5890320718138]
  We propose a red-teaming setting that overlays safety-aligned and harmless UI elements onto screenshots to misdirect the agent's visual grounding.<n>Our method uses a modular Editor-Overlapper-Victim pipeline and an iterative search procedure that samples multiple candidate edits.<n>Our attacks improve attack success rate by up to 4.4x over random injection on the strongest victims.
  arXiv  Detail & Related papers  (2026-04-09T05:32:34Z)
• Silent Egress: When Implicit Prompt Injection Makes LLM Agents Leak Without a Trace [0.0]
  We show that adversarial instructions embedded in automatically generated URL previews can introduce a system-level risk that we refer to as silent egress.<n>Using a fully local and reproducible testbed, we demonstrate that a malicious web page can induce an agent to issue outbound requests that exfiltrate sensitive runtime context.<n>In 480 experimental runs with a qwen2.5:7b-based agent, the attack succeeds with high probability (P (egress) =0.89), and 95% of successful attacks are not detected by output-based safety checks.
  arXiv  Detail & Related papers  (2026-02-25T22:26:23Z)
• SkillJect: Automating Stealthy Skill-Based Prompt Injection for Coding Agents with Trace-Driven Closed-Loop Refinement [120.52289344734415]
  We propose an automated framework for stealthy prompt injection tailored to agent skills.<n>The framework forms a closed loop with three agents: an Attack Agent that synthesizes injection skills under explicit stealth constraints, a Code Agent that executes tasks using the injected skills and an Evaluate Agent that logs action traces.<n>Our method consistently achieves high attack success rates under realistic settings.
  arXiv  Detail & Related papers  (2026-02-15T16:09:48Z)
• Poison Once, Control Anywhere: Clean-Text Visual Backdoors in VLM-based Mobile Agents [54.35629963816521]
  This work introduces VIBMA, the first clean-text backdoor attack targeting VLM-based mobile agents.<n>The attack injects malicious behaviors into the model by modifying only the visual input.<n>We show that our attack achieves high success rates while preserving clean-task behavior.
  arXiv  Detail & Related papers  (2025-06-16T08:09:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.