Related papers: AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents

AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents

URL: http://arxiv.org/abs/2603.23007v1
Date: Tue, 24 Mar 2026 09:51:43 GMT
Title: AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents
Abstract summary: Mobile graphical user interface (GUI) agents autonomously control applications and operating systems (OS)<n>We propose AgentRAE, a novel backdoor attack capable of inducing Remote Action Execution in mobile GUI agents using visually natural triggers.<n>Our evaluation reveals that the proposed backdoor preserves clean performance with an attack success rate of over 90% across ten mobile operations.
Score: 18.82273534480229
License: http://creativecommons.org/licenses/by/4.0/
Abstract: The rapid adoption of mobile graphical user interface (GUI) agents, which autonomously control applications and operating systems (OS), exposes new system-level attack surfaces. Existing backdoors against web GUI agents and general GenAI models rely on environmental injection or deceptive pop-ups to mislead the agent operation. However, these techniques do not work on screenshots-based mobile GUI agents due to the challenges of restricted trigger design spaces, OS background interference, and conflicts in multiple trigger-action mappings. We propose AgentRAE, a novel backdoor attack capable of inducing Remote Action Execution in mobile GUI agents using visually natural triggers (e.g., benign app icons in notifications). To address the underfitting caused by natural triggers and achieve accurate multi-target action redirection, we design a novel two-stage pipeline that first enhances the agent's sensitivity to subtle iconographic differences via contrastive learning, and then associates each trigger with a specific mobile GUI agent action through a backdoor post-training. Our extensive evaluation reveals that the proposed backdoor preserves clean performance with an attack success rate of over 90% across ten mobile operations. Furthermore, it is hard to visibly detect the benign-looking triggers and circumvents eight representative state-of-the-art defenses. These results expose an overlooked backdoor vector in mobile GUI agents, underscoring the need for defenses that scrutinize notification-conditioned behaviors and internal agent representations.

Related papers

A Task-State Representation for Long-Horizon Mobile GUI Agents [53.359524744953575]
Task-State Representation (TSR) is a training-free framework that explicitly decouples task state from sensory input.<n> Experiments across four mobile GUI benchmarks validate TSR's effectiveness, yielding up to a 12 absolute point increase in success rate.
arXiv Detail & Related papers (2026-07-01T06:37:21Z)
(A)I Sees What You Don't: Exploiting New Attack Surfaces in Third-Party Mobile Agents [6.848663048236236]
Third-party mobile agents powered by Vision-Language Models (VLMs) have emerged as a promising paradigm for automating smartphone interactions.<n>In this paper, we summarize key differences between third-party mobile agent apps and general apps when interacting with the environment, analyze the security posture of agents, and identify two unique attack surfaces.<n>We design and implement seven concrete attacks, from subliminal text injection and invisible pixel zone exploitation to screenshot tampering and host PC command injection.
arXiv Detail & Related papers (2026-07-01T02:17:19Z)
WebAgentGuard: A Reasoning-Driven Guard Model for Detecting Prompt Injection Attacks in Web Agents [117.65855863464863]
Web agents are highly vulnerable to prompt injection attacks.<n>Existing defenses, including system prompt defenses and direct fine-tuning of agents, have shown limited effectiveness.<n>We introduce WebAgentGuard, a reasoning-driven, multimodal guard model for prompt injection detection.
arXiv Detail & Related papers (2026-04-14T04:50:35Z)
CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation [68.53387633351484]
We propose CORA (COnformal Risk-controlled GUI Agent), a post-policy, pre-action safeguarding framework that provides statistical guarantees on harmful executed actions.<n> CORA reformulates safety as selective action execution: we train a Guardian model to estimate action-conditional risk for each proposed step.<n>To rigorously evaluate this paradigm, we introduce Phone-Harm, a new benchmark of mobile safety violations with step-level harm labels.
arXiv Detail & Related papers (2026-04-10T09:41:21Z)
Zero-Permission Manipulation: Can We Trust Large Multimodal Model Powered GUI Agents? [6.9619059967556725]
Action Rebinding is a novel attack that allows a seemingly-benign app with zero dangerous permissions to rebind an agent's execution.<n>We weaponize the agent's task-recovery logic and Android's UI state preservation to orchestrate programmable, multi-step attack chains.<n>Our results demonstrate a 100% success rate for atomic action rebinding and the ability to reliably orchestrate multi-step attack chains.
arXiv Detail & Related papers (2026-01-18T10:54:54Z)
It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents [52.81924177620322]
Web-based agents powered by large language models are increasingly used for tasks such as email management or professional networking.<n>Their reliance on dynamic web content makes them vulnerable to prompt injection attacks: adversarial instructions hidden in interface elements that persuade the agent to divert from its original task.<n>We introduce the Task-Redirecting Agent Persuasion Benchmark (TRAP), an evaluation for studying how persuasion techniques misguide autonomous web agents on realistic tasks.
arXiv Detail & Related papers (2025-12-29T01:09:10Z)
VisualTrap: A Stealthy Backdoor Attack on GUI Agents via Visual Grounding Manipulation [73.92237451442752]
This work reveals that the visual grounding of GUI agent-mapping textual plans to GUI elements can introduce vulnerabilities.<n>With backdoor attack targeting visual grounding, the agent's behavior can be compromised even when given correct task-solving plans.<n>We propose VisualTrap, a method that can hijack the grounding by misleading the agent to locate textual plans to trigger locations instead of the intended targets.
arXiv Detail & Related papers (2025-07-09T14:36:00Z)
Poison Once, Control Anywhere: Clean-Text Visual Backdoors in VLM-based Mobile Agents [54.35629963816521]
This work introduces VIBMA, the first clean-text backdoor attack targeting VLM-based mobile agents.<n>The attack injects malicious behaviors into the model by modifying only the visual input.<n>We show that our attack achieves high success rates while preserving clean-task behavior.
arXiv Detail & Related papers (2025-06-16T08:09:32Z)
Hidden Ghost Hand: Unveiling Backdoor Vulnerabilities in MLLM-Powered Mobile GUI Agents [19.348335171985152]
MLLM-powered GUI agents naturally expose multiple interaction-level triggers.<n>We introduce AgentGhost, an effective and stealthy framework for red-teaming backdoor attacks.<n>AgentGhost is effective and generic, with attack accuracy that reaches 99.7% on three attack objectives, and shows stealthiness with only 1% utility degradation.
arXiv Detail & Related papers (2025-05-20T14:29:18Z)
The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injections [21.322212760700957]
A Large Language Model (LLM) powered GUI agent is a specialized autonomous system that performs tasks on the user's behalf according to high-level instructions.<n>To complete real-world tasks, such as filling forms or booking services, GUI agents often need to process and act on sensitive user data.<n>These attacks often exploit the discrepancy between visual saliency for agents and human users.
arXiv Detail & Related papers (2025-04-15T15:21:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.