SlowBA: An efficiency backdoor attack towards VLM-based GUI agents

• URL: http://arxiv.org/abs/2603.08316v2
• Date: Tue, 10 Mar 2026 11:10:35 GMT
• Title: SlowBA: An efficiency backdoor attack towards VLM-based GUI agents
• Abstract summary: We introduce SlowBA, a novel backdoor attack that targets the responsiveness of VLM-based GUI agents.<n>The key idea is to manipulate response latency by inducing excessively long reasoning chains under specific trigger patterns.<n>Experiments demonstrate that SlowBA can significantly increase response length and latency while largely preserving task accuracy.
• Score: 13.613479645526334
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern vision-language-model (VLM) based graphical user interface (GUI) agents are expected not only to execute actions accurately but also to respond to user instructions with low latency. While existing research on GUI-agent security mainly focuses on manipulating action correctness, the security risks related to response efficiency remain largely unexplored. In this paper, we introduce SlowBA, a novel backdoor attack that targets the responsiveness of VLM-based GUI agents. The key idea is to manipulate response latency by inducing excessively long reasoning chains under specific trigger patterns. To achieve this, we propose a two-stage reward-level backdoor injection (RBI) strategy that first aligns the long-response format and then learns trigger-aware activation through reinforcement learning. In addition, we design realistic pop-up windows as triggers that naturally appear in GUI environments, improving the stealthiness of the attack. Extensive experiments across multiple datasets and baselines demonstrate that SlowBA can significantly increase response length and latency while largely preserving task accuracy. The attack remains effective even with a small poisoning ratio and under several defense settings. These findings reveal a previously overlooked security vulnerability in GUI agents and highlight the need for defenses that consider both action correctness and response efficiency. Code can be found in https://github.com/tu-tuing/SlowBA.

Related papers

• AutoDojo: Adaptive Black-Box Attacks Reveal the Limits of IPI Defenses and Task-Specification Effects in LLM Agents [57.34566159148893]
  Indirect prompt injection (IPI) is a major security threat to LLM-powered agents.<n>We develop AutoDojo, an adaptive extension of AgentDojo that optimize IPI against a given defense.
  arXiv  Detail & Related papers  (2026-06-13T02:09:08Z)
• Backdoor Attacks on Open Vocabulary Object Detectors via Multi-Modal Prompt Tuning [5.0734761482919115]
  Open-vocabulary object detectors (OVODs) unify vision and language to detect arbitrary object categories based on text prompts.<n>We conduct the first study of backdoor attacks on OVODs and reveal a new attack surface introduced by prompt tuning.
  arXiv  Detail & Related papers  (2025-11-16T19:05:31Z)
• Visual Backdoor Attacks on MLLM Embodied Decision Making via Contrastive Trigger Learning [89.1856483797116]
  We introduce BEAT, the first framework to inject visual backdoors into MLLM-based embodied agents.<n>Unlike textual triggers, object triggers exhibit wide variation across viewpoints and lighting, making them difficult to implant reliably.<n>BEAT achieves attack success rates up to 80%, while maintaining strong benign task performance.
  arXiv  Detail & Related papers  (2025-10-31T16:50:49Z)
• Realistic Environmental Injection Attacks on GUI Agents [6.38492008798679]
  GUI agents built on LVLMs are increasingly used to interact with websites.<n>Their exposure to open-world content makes them vulnerable to Environmental Injection Attacks (EIAs)<n>We propose Chameleon, an attack framework with two main novelties.
  arXiv  Detail & Related papers  (2025-09-14T12:47:54Z)
• VisualTrap: A Stealthy Backdoor Attack on GUI Agents via Visual Grounding Manipulation [73.92237451442752]
  This work reveals that the visual grounding of GUI agent-mapping textual plans to GUI elements can introduce vulnerabilities.<n>With backdoor attack targeting visual grounding, the agent's behavior can be compromised even when given correct task-solving plans.<n>We propose VisualTrap, a method that can hijack the grounding by misleading the agent to locate textual plans to trigger locations instead of the intended targets.
  arXiv  Detail & Related papers  (2025-07-09T14:36:00Z)
• Poison Once, Control Anywhere: Clean-Text Visual Backdoors in VLM-based Mobile Agents [54.35629963816521]
  This work introduces VIBMA, the first clean-text backdoor attack targeting VLM-based mobile agents.<n>The attack injects malicious behaviors into the model by modifying only the visual input.<n>We show that our attack achieves high success rates while preserving clean-task behavior.
  arXiv  Detail & Related papers  (2025-06-16T08:09:32Z)
• Hidden Ghost Hand: Unveiling Backdoor Vulnerabilities in MLLM-Powered Mobile GUI Agents [19.348335171985152]
  MLLM-powered GUI agents naturally expose multiple interaction-level triggers.<n>We introduce AgentGhost, an effective and stealthy framework for red-teaming backdoor attacks.<n>AgentGhost is effective and generic, with attack accuracy that reaches 99.7% on three attack objectives, and shows stealthiness with only 1% utility degradation.
  arXiv  Detail & Related papers  (2025-05-20T14:29:18Z)
• EVA: Red-Teaming GUI Agents via Evolving Indirect Prompt Injection [14.83331240126743]
  multimodal agents are increasingly trained to operate graphical user interfaces (GUIs) to complete user tasks.<n>We propose EVA, a framework for indirect prompt injection, which transforms the attack into a closed loop optimization.<n>We evaluate EVA on six widely used generalist and specialist GUI agents in realistic settings such as popup manipulation, chat based phishing, payments, and email composition.
  arXiv  Detail & Related papers  (2025-05-20T12:41:05Z)
• MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents [60.30753230776882]
  LLM agents are vulnerable to indirect prompt injection (IPI) attacks, where malicious tasks embedded in tool-retrieved information can redirect the agent to take unauthorized actions.<n>We present MELON, a novel IPI defense that detects attacks by re-executing the agent's trajectory with a masked user prompt modified through a masking function.
  arXiv  Detail & Related papers  (2025-02-07T18:57:49Z)
• Dissecting Adversarial Robustness of Multimodal LM Agents [70.2077308846307]
  We manually create 200 targeted adversarial tasks and evaluation scripts in a realistic threat model on top of VisualWebArena.<n>We find that we can successfully break latest agents that use black-box frontier LMs, including those that perform reflection and tree search.<n>We also use ARE to rigorously evaluate how the robustness changes as new components are added.
  arXiv  Detail & Related papers  (2024-06-18T17:32:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.