Defending Wearable VLMs Against Private Attribute Inference

• URL: http://arxiv.org/abs/2608.28691v1
• Date: Wed, 26 Aug 2026 21:36:45 GMT
• Title: Defending Wearable VLMs Against Private Attribute Inference
• Abstract summary: Wearable VLM pipelines promise continuous multimodal assistance from egocentric visual capture.<n>The same egocentric evidence needed for useful assistance can also reveal private attributes about the wearer or nearby bystanders.<n>We propose Token-Guided Attribute Privacy (TGAP), a pre-LLM token disentangler that learns a residual transformation of visual tokens before they leave the trusted boundary.
• Score: 21.179736294173168
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Wearable VLM pipelines promise continuous multimodal assistance from egocentric visual capture: a user asks a task-driven question about the surrounding scene, and the system uses compact visual tokens to support language reasoning. The challenge motivating this work is that the same egocentric evidence needed for useful assistance can also reveal private attributes about the wearer or nearby bystanders. We investigate this as a joint privacy-utility problem for split VLM inference, where visual encoding occurs within a trusted device boundary but intermediate visual tokens may be transmitted to downstream reasoning components. This exposes an understudied leakage surface: even when final textual responses are benign, external attackers or untrusted downstream components can recover private attributes from transmitted visual tokens. To evaluate this tension, we construct a paired privacy-utility benchmark with 3,221 image-question records, each paired with a utility question and privacy labels covering location, income, sex, and interests. We further propose Token-Guided Attribute Privacy (TGAP), a pre-LLM token disentangler that learns a residual transformation of visual tokens before they leave the trusted boundary. TGAP combines utility preservation, identity regularization, semantic privacy suppression, and image-driven representation suppression, avoiding the utility loss caused by coarse hard or attention masking. On the benchmark used for source-model evaluation, TGAP reduces privacy accuracy from 56.7\% to 7.4\%, a 49.3\% absolute drop, while maintaining relaxed utility at 74.4\%. These results suggest that securing the compact token interface is a practical path toward privacy-preserving wearable multimodal AI.

Related papers

• Don't Send What You Don't Need: Question-Guided Token Pruning as a Privacy Defense for Vision-Language Models [0.30938904602244355]
  QPriv-VL is a question-guided, privacy-aware token-pruning framework.<n>It prunes visual tokens before transmission based on task utility and privacy sensitivity.<n>We evaluate it on GQA, OK-VQA, VQAv2, SLAKE, VQA-RAD, and PathVQA against four privacy attack families.
  arXiv  Detail & Related papers  (2026-09-14T14:48:39Z)
• Privacy-Aware Video Anomaly Detection through Orthogonal Subspace Projection [14.631956899935721]
  Video anomaly detection (VAD) systems often prioritize accuracy while overlooking privacy concerns, limiting their suitability for real-world deployment.<n>We propose the Orthogonal Projection Layer (OPL), a lightweight module that removes task-irrelevant variations to produce representations focused on anomaly-relevant cues.<n>To address privacy risks in human-centered scenarios, we introduce Guided OPL (G-OPL), which suppresses facial attributes using weak supervision from face-presence signals while preserving non-identifying features such as pose and motion.
  arXiv  Detail & Related papers  (2026-05-09T03:46:39Z)
• SIDeR: Semantic Identity Decoupling for Unrestricted Face Privacy [53.75084833636302]
  We propose SIDeR, a Semantic decoupling-driven framework for unrestricted face privacy protection.<n> SIDeR decomposes a facial image into a machine-recognizable identity feature vector and a visually perceptible semantic appearance component.<n>For authorized access, SIDeR can be restored to its original form when the correct password is provided.
  arXiv  Detail & Related papers  (2026-02-04T19:30:48Z)
• Who Can See Through You? Adversarial Shielding Against VLM-Based Attribute Inference Attacks [13.326888254423901]
  VLM-based attribute inference attacks have emerged as a serious privacy concern, enabling adversaries to infer private attributes from images shared on social media.<n>We propose a novel protection method that jointly optimize privacy suppression and utility preservation under a visual consistency constraint.<n>Our method effectively reduces PAR below 25%, keeps NPAR above 88%, and generalizes well to unseen and paraphrased privacy questions.
  arXiv  Detail & Related papers  (2025-12-20T08:08:50Z)
• Privacy Beyond Pixels: Latent Anonymization for Privacy-Preserving Video Understanding [56.369026347458835]
  We introduce a novel formulation of visual privacy preservation for video foundation models that operates entirely in the latent space.<n>Current privacy preservation methods on input-pixel-level anonymization require retraining the entire utility video model.<n>A lightweight Anonym Adapter Module (AAM) removes private information from video features while retaining general task utility.
  arXiv  Detail & Related papers  (2025-11-11T18:56:27Z)
• VoxGuard: Evaluating User and Attribute Privacy in Speech via Membership Inference Attacks [51.68795949691009]
  We introduce VoxGuard, a framework grounded in differential privacy and membership inference.<n>For attributes, we show that simple transparent attacks recover gender and accent with near-perfect accuracy even after anonymization.<n>Our results demonstrate that EER substantially underestimates leakage, highlighting the need for low-FPR evaluation.
  arXiv  Detail & Related papers  (2025-09-22T20:57:48Z)
• DP-Fusion: Token-Level Differentially Private Inference for Large Language Models [51.71591819896191]
  Large language models (LLMs) do not preserve privacy at inference-time.<n> DP-Fusion provably bounds the influence a set of tokens in the context can have on the LLM's output.<n>We show that our method creates token-level provably privatized documents with substantially improved theoretical and empirical privacy.
  arXiv  Detail & Related papers  (2025-07-06T20:49:39Z)
• Privacy-Preserving in Connected and Autonomous Vehicles Through Vision to Text Transformation [0.9831489366502302]
  This paper introduces a novel privacy-preserving framework that leverages feedback-based reinforcement learning (RL) and vision-language models (VLMs)<n>The main idea is to convert images into semantically equivalent textual descriptions, ensuring that scene-relevant information is retained while visual privacy is preserved.<n> Evaluation results demonstrate significant improvements in both privacy protection and textual quality.
  arXiv  Detail & Related papers  (2025-06-18T20:02:24Z)
• PersGuard: Preventing Malicious Personalization via Backdoor Attacks on Pre-trained Text-to-Image Diffusion Models [51.458089902581456]
  We introduce PersGuard, a novel backdoor-based approach that prevents malicious personalization of specific images.<n>Our method significantly outperforms existing techniques, offering a more robust solution for privacy and copyright protection.
  arXiv  Detail & Related papers  (2025-02-22T09:47:55Z)
• Subverting Privacy-Preserving GANs: Hiding Secrets in Sanitized Images [13.690485523871855]
  State-of-the-art approaches use privacy-preserving generative adversarial networks (PP-GANs) to enable reliable facial expression recognition without leaking users' identity. We show that it is possible to hide the sensitive identification data in the sanitized output images of such PP-GANs for later extraction.
  arXiv  Detail & Related papers  (2020-09-19T19:02:17Z)
• Privacy-Preserving Image Features via Adversarial Affine Subspace Embeddings [72.68801373979943]
  Many computer vision systems require users to upload image features to the cloud for processing and storage. We propose a new privacy-preserving feature representation. Compared to the original features, our approach makes it significantly more difficult for an adversary to recover private information.
  arXiv  Detail & Related papers  (2020-06-11T17:29:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.