An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks
- URL: http://arxiv.org/abs/2609.09404v1
- Date: Tue, 08 Sep 2026 20:00:03 GMT
- Title: An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks
- Abstract summary: We present MMPIBench, a reproducible benchmark that measures what happens next.<n>It delivers a fixed set of attacks through six visual carriers (OCR text, overlays, EXIF metadata, QR codes, fake interfaces, and hybrids)<n>Attacks complete in approximately 1% of runs but are attempted in 12.8%, and the gap is closed almost entirely at the planning step.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services. Most of these agents also read images, which gives an attacker a way to put text into the agent's context without going through the user. We present MMPIBench, a reproducible benchmark that measures what happens next. It delivers a fixed set of attacks through six visual carriers (OCR text, overlays, EXIF metadata, QR codes, fake interfaces, and hybrids) and records how far each injected instruction travels through the agent, from perception through planning to the tool call. Across 720 runs covering six frameworks, five foundation models, six carriers, and four attacker objectives, attacks complete in approximately 1% of runs but are attempted in 12.8%, and the gap is closed almost entirely at the planning step, where the model reads the injected instruction and declines to act on it. The model matters far more than the framework for whether an instruction is acted on. One model never attempts an attack and recognizes the injection in 59.7% of runs, while two others attempt in 23.6%. We then extend the benchmark to audio, the only other raw perceptual channel current frontier models accept. Only two of the five models ingest audio and only three of the six frameworks deliver it, but where the signal arrives the attack completes in 49% of cells, and in 75% for one model. Reporting completion alone therefore understates exposure, and perceptual channels beyond vision are narrower but much less defended.
Related papers
- AgentAudit: An Open, Extensible Framework for Full-Lifecycle Trust Evaluation of AI Agents [2.8000808592516293]
AgentAudit evaluates the entire execution trace across ten capability, grounding, security and behavioural dimensions.<n>It reads only the recorded execution trace and does not interfere with how the agent runs.<n>We evaluate five language models (OpenAI GPT-5, Claude Sonnet 5, Sarvam 105B, Llama 3.3 70B and Gemini 2.5 Flash) across nine capability and adversarial tasks.
arXiv Detail & Related papers (2026-09-09T08:29:16Z) - Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection [40.37588773528427]
We present Repeat-After-Me, a black-box adaptive visual prompt injection attack that can reveal personally identifiable information or make malicious tool calls.<n>In our evaluation, injections optimized on one surrogate retain 43-46% of the original ASR on two commercial victims.<n>In a default OpenClaw Discord deployment, an untrusted user can use a minimally injected image to overwrite TOOLS.md, enabling future sensitive behaviors like remote code execution and secret exfiltration.
arXiv Detail & Related papers (2026-09-03T22:44:45Z) - Actions Speak Louder than Words: Measuring Cross-Lingual Policy Retention in Tool-Using Agents [22.30468215041597]
A tool-using agent is given the same task in a different language, does it still take the same steps?<n>We make the action policy the measured object across 8 models, 6 parallel benchmarks and 41 languages.
arXiv Detail & Related papers (2026-08-11T16:18:34Z) - MM-ToolSandBox: A Unified Framework for Evaluating Visual Tool-Calling Agents [50.27505102982638]
We introduce MM-ToolSandBox, a benchmark and evaluation framework for visually grounded tool-calling agents.<n>The framework provides a stateful execution environment spanning 500+ tools across 16 application domains.<n>It supports multi-image, multi-turn tasks where agents must ground progressively arriving visual inputs into executable tool calls.
arXiv Detail & Related papers (2026-07-13T17:13:09Z) - VisualClaw: A Real-Time, Personalized Agent for the Physical World [76.89211120690028]
We present VisualClaw, a self-evolving multimodal agent built around two principles.<n>First, hybrid encoding reduces deployment cost by filtering less informative streaming frames with a cascaded gate.<n>Second, skill evolution lets the agent learn from failures, producing skill-bank updates that help future questions.
arXiv Detail & Related papers (2026-06-15T06:58:22Z) - ClawArena: Benchmarking AI Agents in Evolving Information Environments [61.664633997138004]
ClawArena is a benchmark for evaluating AI agents in evolving information environments.<n>Each scenario maintains a complete hidden ground truth while exposing the agent only to noisy, partial, and sometimes contradictory traces.<n> evaluation is organized around three coupled challenges: multi-source conflict reasoning, dynamic belief revision, and implicit personalization.
arXiv Detail & Related papers (2026-04-05T17:55:23Z) - How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition [48.32744727426218]
LLM based agents are increasingly deployed in high stakes settings where they process external data sources such as emails, documents, and code repositories.<n>This creates exposure to indirect prompt injection attacks, where adversarial instructions embedded in external content manipulate agent behavior without user awareness.<n>We present findings from a large scale public red teaming competition evaluating this dual objective across three agent settings.
arXiv Detail & Related papers (2026-03-16T14:49:36Z) - Penetration Testing of Agentic AI: A Comparative Security Analysis Across Models and Frameworks [0.0]
Agentic AI introduces security vulnerabilities that traditional LLM safeguards fail to address.<n>We conduct the first systematic testing and comparative evaluation of agentic AI systems.<n>We identify six distinct defensive behavior patterns including a novel "hallucinated compliance" strategy.
arXiv Detail & Related papers (2025-12-16T19:22:50Z) - Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks [11.371490212283383]
Code-capable large language model (LLM) agents are embedded into software engineering where they can read, write, and execute code.<n>We present JAWS-BENCH, a benchmark spanning three escalating workspaces that mirror attacker capability.<n>We find that under prompt-only conditions in JAWS-0, code agents accept 61% of attacks on average; 58% are harmful, 52% parse, and 27% run end-to-end.
arXiv Detail & Related papers (2025-10-01T18:38:20Z) - Backdoor Attacks Against Speech Language Models [63.07317091368079]
We present the first systematic study of audio backdoor attacks against speech language models.<n>We demonstrate its effectiveness across four speech encoders and three datasets, covering four tasks.<n>We propose a fine-tuning-based defense that mitigates the threat of poisoned pretrained encoders.
arXiv Detail & Related papers (2025-10-01T17:45:04Z) - Dissecting Adversarial Robustness of Multimodal LM Agents [70.2077308846307]
We manually create 200 targeted adversarial tasks and evaluation scripts in a realistic threat model on top of VisualWebArena.<n>We find that we can successfully break latest agents that use black-box frontier LMs, including those that perform reflection and tree search.<n>We also use ARE to rigorously evaluate how the robustness changes as new components are added.
arXiv Detail & Related papers (2024-06-18T17:32:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.