Assessing Privacy Preservation and Utility in Online Vision-Language Models

• URL: http://arxiv.org/abs/2604.09695v1
• Date: Mon, 06 Apr 2026 20:44:46 GMT
• Title: Assessing Privacy Preservation and Utility in Online Vision-Language Models
• Abstract summary: Images contain relationships that relate to Personally Identifiable Information (PII)<n>We investigate how the extraction of contextual relationships from images can lead to direct (explicit) or indirect (implicit) exposure of PII.<n>We propose methods to protect privacy while preserving the intended utility of the images in Vision Language Model (VLM)-based applications.
• Score: 15.74713639305756
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The increasing use of Online Vision Language Models (OVLMs) for processing images has introduced significant privacy risks, as individuals frequently upload images for various utilities, unaware of the potential for privacy violations. Images contain relationships that relate to Personally Identifiable Information (PII), where even seemingly harmless details can indirectly reveal sensitive information through surrounding clues. This paper explores the critical issue of PII disclosure in images uploaded to OVLMs and its implications for user privacy. We investigate how the extraction of contextual relationships from images can lead to direct (explicit) or indirect (implicit) exposure of PII, significantly compromising personal privacy. Furthermore, we propose methods to protect privacy while preserving the intended utility of the images in Vision Language Model (VLM)-based applications. Our evaluation demonstrates the efficacy of these techniques, highlighting the delicate balance between maintaining utility and protecting privacy in online image processing environments. Index Terms-Personally Identifiable Information (PII), Privacy, Utility, privacy concerns, sensitive information

Related papers

• Vision Language Model Helps Private Information De-Identification in Vision Data [55.425628316813174]
  We introduce VisShield, an end-to-end framework designed to enhance the privacy awareness of Visual Language Models (VLMs)<n>Our framework consists of two key components: a specialized instruction-tuning dataset OPTIC and a tailored training methodology.<n>Our approach ensures that VLMs recognize privacy-sensitive text and output precise bounding boxes for detected entities.
  arXiv  Detail & Related papers  (2026-06-08T07:30:20Z)
• Privacy-Preserving in Connected and Autonomous Vehicles Through Vision to Text Transformation [0.9831489366502302]
  This paper introduces a novel privacy-preserving framework that leverages feedback-based reinforcement learning (RL) and vision-language models (VLMs)<n>The main idea is to convert images into semantically equivalent textual descriptions, ensuring that scene-relevant information is retained while visual privacy is preserved.<n> Evaluation results demonstrate significant improvements in both privacy protection and textual quality.
  arXiv  Detail & Related papers  (2025-06-18T20:02:24Z)
• Image Privacy Protection: A Survey [32.020322218775526]
  Images serve as a crucial medium for communication, presenting information in a visually engaging format that facilitates rapid comprehension of key points.<n>If not managed properly, this information may be vulnerable to exploitation for personal gain, potentially infringing on privacy rights and other legal entitlements.<n>Existing reviews tend to categorize either by specific scenarios, or by specific privacy objectives.
  arXiv  Detail & Related papers  (2024-12-05T08:09:25Z)
• Activity Recognition on Avatar-Anonymized Datasets with Masked Differential Privacy [64.32494202656801]
  Privacy-preserving computer vision is an important emerging problem in machine learning and artificial intelligence.<n>We present anonymization pipeline that replaces sensitive human subjects in video datasets with synthetic avatars within context.<n>We also proposeMaskDP to protect non-anonymized but privacy sensitive background information.
  arXiv  Detail & Related papers  (2024-10-22T15:22:53Z)
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action [54.11479432110771]
  PrivacyLens is a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories.<n>We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds.<n>State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions.
  arXiv  Detail & Related papers  (2024-08-29T17:58:38Z)
• Privacy-preserving Optics for Enhancing Protection in Face De-identification [60.110274007388135]
  We propose a hardware-level face de-identification method to solve this vulnerability. We also propose an anonymization framework that generates a new face using the privacy-preserving image, face heatmap, and a reference face image from a public dataset as input.
  arXiv  Detail & Related papers  (2024-03-31T19:28:04Z)
• Diff-Privacy: Diffusion-based Face Privacy Protection [58.1021066224765]
  In this paper, we propose a novel face privacy protection method based on diffusion models, dubbed Diff-Privacy. Specifically, we train our proposed multi-scale image inversion module (MSI) to obtain a set of SDM format conditional embeddings of the original image. Based on the conditional embeddings, we design corresponding embedding scheduling strategies and construct different energy functions during the denoising process to achieve anonymization and visual identity information hiding.
  arXiv  Detail & Related papers  (2023-09-11T09:26:07Z)
• Toward Privacy and Utility Preserving Image Representation [26.768476643200664]
  We study the novel problem of creating privacy-preserving image representations with respect to a given utility task. We propose a principled framework called the Adversarial Image Anonymizer (AIA) AIA first creates an image representation using a generative model, then enhances the learned image representations using adversarial learning to preserve privacy and utility for a given task.
  arXiv  Detail & Related papers  (2020-09-30T01:25:00Z)
• Subverting Privacy-Preserving GANs: Hiding Secrets in Sanitized Images [13.690485523871855]
  State-of-the-art approaches use privacy-preserving generative adversarial networks (PP-GANs) to enable reliable facial expression recognition without leaking users' identity. We show that it is possible to hide the sensitive identification data in the sanitized output images of such PP-GANs for later extraction.
  arXiv  Detail & Related papers  (2020-09-19T19:02:17Z)
• InfoScrub: Towards Attribute Privacy by Targeted Obfuscation [77.49428268918703]
  We study techniques that allow individuals to limit the private information leaked in visual data. We tackle this problem in a novel image obfuscation framework. We find our approach generates obfuscated images faithful to the original input images, and additionally increase uncertainty by 6.2$times$ (or up to 0.85 bits) over the non-obfuscated counterparts.
  arXiv  Detail & Related papers  (2020-05-20T19:48:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.