Anonymization-Enhanced Privacy Protection for Mobile GUI Agents: Available but Invisible

• URL: http://arxiv.org/abs/2602.10139v2
• Date: Sat, 14 Feb 2026 04:21:20 GMT
• Title: Anonymization-Enhanced Privacy Protection for Mobile GUI Agents: Available but Invisible
• Authors: Lepeng Zhao, Zhenhua Zou, Shuo Li, Zhuotao Liu,
• Abstract summary: Mobile Graphical User Interface (GUI) agents have demonstrated strong capabilities in automating complex smartphone tasks.<n>We propose anonymization-based privacy protection framework that enforces the principle of available-but-invisible access to sensitive data.<n>Our system detects sensitive UI content using a PII-aware recognition model and replaces it with deterministic, type-preserving placeholders.
• Score: 12.742325129012576
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Mobile Graphical User Interface (GUI) agents have demonstrated strong capabilities in automating complex smartphone tasks by leveraging multimodal large language models (MLLMs) and system-level control interfaces. However, this paradigm introduces significant privacy risks, as agents typically capture and process entire screen contents, thereby exposing sensitive personal data such as phone numbers, addresses, messages, and financial information. Existing defenses either reduce UI exposure, obfuscate only task-irrelevant content, or rely on user authorization, but none can protect task-critical sensitive information while preserving seamless agent usability. We propose an anonymization-based privacy protection framework that enforces the principle of available-but-invisible access to sensitive data: sensitive information remains usable for task execution but is never directly visible to the cloud-based agent. Our system detects sensitive UI content using a PII-aware recognition model and replaces it with deterministic, type-preserving placeholders (e.g., PHONE_NUMBER#a1b2c) that retain semantic categories while removing identifying details. A layered architecture comprising a PII Detector, UI Transformer, Secure Interaction Proxy, and Privacy Gatekeeper ensures consistent anonymization across user instructions, XML hierarchies, and screenshots, mediates all agent actions over anonymized interfaces, and supports narrowly scoped local computations when reasoning over raw values is necessary. Extensive experiments on the AndroidLab and PrivScreen benchmarks show that our framework substantially reduces privacy leakage across multiple models while incurring only modest utility degradation, achieving the best observed privacy-utility trade-off among existing methods. Code available at: https://github.com/one-step-beh1nd/gui_privacy_protection

Related papers

• Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs [61.15237978606501]
  Large language models can infer private user attributes from user-generated text.<n>Existing anonymization-based defenses are coarse-grained, lacking word-level precision in anonymizing privacy-leaking elements.<n>We propose a unified defense framework that combines fine-grained anonymization (TRACE) with inference-preventing optimization (RPS)
  arXiv  Detail & Related papers  (2026-02-12T03:37:50Z)
• GUIGuard: Toward a General Framework for Privacy-Preserving GUI Agents [38.42792282309646]
  GUIs expose richer, more accessible private information, and privacy risks depend on interaction trajectories across sequential scenes.<n>We propose a three-stage framework for privacy-preserving GUI agents: privacy recognition, privacy protection, and task execution under protection.<n>Our results highlight privacy recognition as a critical bottleneck for practical GUI agents.
  arXiv  Detail & Related papers  (2026-01-26T11:33:40Z)
• Privacy Beyond Pixels: Latent Anonymization for Privacy-Preserving Video Understanding [56.369026347458835]
  We introduce a novel formulation of visual privacy preservation for video foundation models that operates entirely in the latent space.<n>Current privacy preservation methods on input-pixel-level anonymization require retraining the entire utility video model.<n>A lightweight Anonym Adapter Module (AAM) removes private information from video features while retaining general task utility.
  arXiv  Detail & Related papers  (2025-11-11T18:56:27Z)
• Effective and Stealthy One-Shot Jailbreaks on Deployed Mobile Vision-Language Agents [29.62914440645731]
  We present a one-shot jailbreak attack that leverages in-app prompt injections.<n> malicious apps embed short prompts in UI text that remain inert during human interaction but are revealed when an agent drives the UI via ADB.<n>Our framework comprises three crucial components: (1) low-privilege perception-chain targeting, which injects payloads into malicious apps as the agent's visual inputs; (2) user-invisible activation, a touch-based trigger that discriminates agent from human touches using physical touch attributes and exposes the payload only during agent operation; and (3) one-shot prompt efficacy, a stealthy-guided, character-level
  arXiv  Detail & Related papers  (2025-10-09T05:34:57Z)
• GAMA: A General Anonymizing Multi-Agent System for Privacy Preservation Enhanced by Domain Rules and Disproof Mechanism [14.491054279033968]
  General Anonymizing Multi-Agent System (GAMA)<n>GAMA divides the agents' workspace into private and public spaces, ensuring privacy through a structured anonymization mechanism.<n>We evaluate GAMA on two general question-answering datasets, a public privacy leakage benchmark, and two customized question-answering datasets related to privacy.
  arXiv  Detail & Related papers  (2025-09-12T07:22:49Z)
• DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents [52.92354372596197]
  Large Language Models (LLMs) are increasingly central to agentic systems due to their strong reasoning and planning capabilities.<n>This interaction also introduces the risk of prompt injection attacks, where malicious inputs from external sources can mislead the agent's behavior.<n>We propose a Dynamic Rule-based Isolation Framework for Trustworthy agentic systems, which enforces both control and data-level constraints.
  arXiv  Detail & Related papers  (2025-06-13T05:01:09Z)
• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
  differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit. We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
  arXiv  Detail & Related papers  (2024-06-20T13:54:32Z)
• Can Language Models be Instructed to Protect Personal Information? [30.187731765653428]
  We introduce PrivQA -- a benchmark to assess the privacy/utility trade-off when a model is instructed to protect specific categories of personal information in a simulated scenario. We find that adversaries can easily circumvent these protections with simple jailbreaking methods through textual and/or image inputs. We believe PrivQA has the potential to support the development of new models with improved privacy protections, as well as the adversarial robustness of these protections.
  arXiv  Detail & Related papers  (2023-10-03T17:30:33Z)
• Diff-Privacy: Diffusion-based Face Privacy Protection [58.1021066224765]
  In this paper, we propose a novel face privacy protection method based on diffusion models, dubbed Diff-Privacy. Specifically, we train our proposed multi-scale image inversion module (MSI) to obtain a set of SDM format conditional embeddings of the original image. Based on the conditional embeddings, we design corresponding embedding scheduling strategies and construct different energy functions during the denoising process to achieve anonymization and visual identity information hiding.
  arXiv  Detail & Related papers  (2023-09-11T09:26:07Z)
• SPAct: Self-supervised Privacy Preservation for Action Recognition [73.79886509500409]
  Existing approaches for mitigating privacy leakage in action recognition require privacy labels along with the action labels from the video dataset. Recent developments of self-supervised learning (SSL) have unleashed the untapped potential of the unlabeled data. We present a novel training framework which removes privacy information from input video in a self-supervised manner without requiring privacy labels.
  arXiv  Detail & Related papers  (2022-03-29T02:56:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.