If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systems

• URL: http://arxiv.org/abs/2604.19844v1
• Date: Tue, 21 Apr 2026 11:27:30 GMT
• Title: If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systems
• Abstract summary: Environmental signals are essential in-band signals that should influence agent behavior.<n>Similar signals could also be crafted to operate as misleading visual injections.<n>Current LVLM-based agents fail to reliably balance this trade-off.<n>We propose a multi-agent defense framework that separates perception from decision-making.
• Score: 23.899383110296622
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in embodied Vision-Language Agentic Systems (VLAS), powered by large vision-language models (LVLMs), enable AI systems to perceive and reason over real-world scenes. Within this context, environmental signals such as traffic lights are essential in-band signals that can and should influence agent behavior. However, similar signals could also be crafted to operate as misleading visual injections, overriding user intent and posing security risks. This duality creates a fundamental challenge: agents must respond to legitimate environmental cues while remaining robust to misleading ones. We refer to this tension as trust boundary confusion. To study this behavior, we design a dual-intent dataset and evaluation framework, through which we show that current LVLM-based agents fail to reliably balance this trade-off, either ignoring useful signals or following harmful ones. We systematically evaluate 7 LVLM agents across multiple embodied settings under both structure-based and noise-based visual injections. To address these vulnerabilities, we propose a multi-agent defense framework that separates perception from decision-making to dynamically assess the reliability of visual inputs. Our approach significantly reduces misleading behaviors while preserving correct responses and provides robustness guarantees under adversarial perturbations. The code of the evaluation framework and artifacts are made available at https://anonymous.4open.science/r/Visual-Prompt-Inject.

Related papers

• Automatic Red Teaming for Implicit Vulnerabilities of Text-to-Image Models [58.59008920408855]
  We propose Adversarial Probing for Implicit VulnErabilities (AdvPIE) to expose implicit vulnerabilities without requiring access to the parameters of target models.<n> AdvPIE adopts a policy agent to generate and refine implicit adversarial prompts based on the feedback from a judge agent.<n>Experiments on standard and safety-aligned T2I models show that AdvPIE1 effectively uncovers implicit vulnerabilities.
  arXiv  Detail & Related papers  (2026-09-05T13:45:09Z)
• SAVeS: Steering Safety Judgments in Vision-Language Models via Semantic Cues [50.928203120795786]
  We study whether multimodal safety behavior in vision-language models can be steered by simple semantic cues.<n>We introduce a semantic steering framework that applies controlled textual, visual, and cognitive interventions without changing the underlying scene content.
  arXiv  Detail & Related papers  (2026-03-19T16:18:00Z)
• HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task [42.665798473119516]
  We propose an architecture-agnostic safeguard featuring Context-Guided Chain-of-Thought (CG-CoT)<n>CG-CoT decomposes risk assessment into active perception that sequentially anchors attention to interaction targets and relevant spatial neighborhoods.<n> Experiments demonstrate that our model HomeGuard significantly enhances safety, improving risk match rates by over 30% compared to base models.
  arXiv  Detail & Related papers  (2026-03-15T13:09:43Z)
• ReViP: Reducing False Completion in Vision-Language-Action Models with Vision-Proprioception Rebalance [50.05984919728878]
  We present ReViP, a novel VLA framework with Vision-Proprioception Rebalance to enhance visual grounding and robustness under perturbations.<n>Specifically, we use an external VLM as a task-stage observer to extract real-time task-centric visual cues from visual observations.<n>To evaluate false completion, we propose the first False-Completion Benchmark Suite built on LIBERO with controlled settings such as Object-Drop.
  arXiv  Detail & Related papers  (2026-01-23T11:31:07Z)
• Reasoning-Style Poisoning of LLM Agents via Stealthy Style Transfer: Process-Level Attacks and Runtime Monitoring in RSV Space [4.699272847316498]
  Reasoning-Style Poisoning (RSP) manipulates how agents process information rather than what they process.<n>Generative Style Injection (GSI) rewrites retrieved documents into pathological tones.<n>RSP-M is a lightweight runtime monitor that calculates RSV metrics in real-time and triggers alerts when values exceed safety thresholds.
  arXiv  Detail & Related papers  (2025-12-16T14:34:10Z)
• AgentSight: System-Level Observability for AI Agents Using eBPF [10.37440633887049]
  Existing tools observe either an agent's high-level intent (via LLM prompts) or its low-level actions (e.g., system calls) but cannot correlate these two views.<n>We introduce AgentSight, an AgentOps observability framework that bridges this semantic gap using a hybrid approach.<n>AgentSight intercepts TLS-encrypted LLM traffic to extract semantic intent, monitors kernel events to observe system-wide effects, and causally correlates these two streams across process boundaries.
  arXiv  Detail & Related papers  (2025-08-02T01:43:39Z)
• Poison Once, Control Anywhere: Clean-Text Visual Backdoors in VLM-based Mobile Agents [54.35629963816521]
  This work introduces VIBMA, the first clean-text backdoor attack targeting VLM-based mobile agents.<n>The attack injects malicious behaviors into the model by modifying only the visual input.<n>We show that our attack achieves high success rates while preserving clean-task behavior.
  arXiv  Detail & Related papers  (2025-06-16T08:09:32Z)
• DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents [52.92354372596197]
  Large Language Models (LLMs) are increasingly central to agentic systems due to their strong reasoning and planning capabilities.<n>This interaction also introduces the risk of prompt injection attacks, where malicious inputs from external sources can mislead the agent's behavior.<n>We propose a Dynamic Rule-based Isolation Framework for Trustworthy agentic systems, which enforces both control and data-level constraints.
  arXiv  Detail & Related papers  (2025-06-13T05:01:09Z)
• Deceptive Automated Interpretability: Language Models Coordinating to Fool Oversight Systems [0.0]
  We show that language models can generate deceptive explanations that evade detection.<n>Our agents employ steganographic methods to hide information in seemingly innocent explanations.<n>All tested LLM agents were capable of deceiving the overseer while achieving high interpretability scores comparable to those of reference labels.
  arXiv  Detail & Related papers  (2025-04-10T15:07:10Z)
• Compromising Embodied Agents with Contextual Backdoor Attacks [69.71630408822767]
  Large language models (LLMs) have transformed the development of embodied intelligence. This paper uncovers a significant backdoor security threat within this process. By poisoning just a few contextual demonstrations, attackers can covertly compromise the contextual environment of a black-box LLM.
  arXiv  Detail & Related papers  (2024-08-06T01:20:12Z)
• Dissecting Adversarial Robustness of Multimodal LM Agents [70.2077308846307]
  We manually create 200 targeted adversarial tasks and evaluation scripts in a realistic threat model on top of VisualWebArena.<n>We find that we can successfully break latest agents that use black-box frontier LMs, including those that perform reflection and tree search.<n>We also use ARE to rigorously evaluate how the robustness changes as new components are added.
  arXiv  Detail & Related papers  (2024-06-18T17:32:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.