BIV-Priv-Seg: Locating Private Content in Images Taken by People With Visual Impairments

• URL: http://arxiv.org/abs/2407.18243v3
• Date: Fri, 10 Jan 2025 15:37:27 GMT
• Title: BIV-Priv-Seg: Locating Private Content in Images Taken by People With Visual Impairments
• Authors: Yu-Yun Tseng, Tanusree Sharma, Lotus Zhang, Abigale Stangl, Leah Findlater, Yang Wang, Danna Gurari,
• Abstract summary: BIV-Priv-Seg is the first dataset originating from people with visual impairments that shows private content.<n>It contains 1,028 images with segmentation annotations for 16 private object categories.<n>We evaluate modern models' performance for locating private content in the dataset.
• Score: 25.365045519494874
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Individuals who are blind or have low vision (BLV) are at a heightened risk of sharing private information if they share photographs they have taken. To facilitate developing technologies that can help them preserve privacy, we introduce BIV-Priv-Seg, the first localization dataset originating from people with visual impairments that shows private content. It contains 1,028 images with segmentation annotations for 16 private object categories. We first characterize BIV-Priv-Seg and then evaluate modern models' performance for locating private content in the dataset. We find modern models struggle most with locating private objects that are not salient, small, and lack text as well as recognizing when private content is absent from an image. We facilitate future extensions by sharing our new dataset with the evaluation server at https://vizwiz.org/tasks-and-datasets/object-localization.

Related papers

• Beyond Blanket Masking: Examining Granularity for Privacy Protection in Images Captured by Blind and Low Vision Users [23.61740342584077]
  We propose FiGPriv, a fine-grained privacy protection framework that selectively masks only high-risk private information.<n>Our approach integrates fine-grained segmentation with a data-driven risk scoring mechanism.<n>We evaluate our framework using the BIV-Priv-Seg dataset and show that FiG-Priv preserves +26% of image content.
  arXiv  Detail & Related papers  (2025-08-12T17:56:36Z)
• Controlling What You Share: Assessing Language Model Adherence to Privacy Preferences [73.5779077857545]
  We build a framework where a local model uses these instructions to rewrite queries, only hiding details deemed sensitive by the user, before sending them to an external model.<n>Experiments with lightweight local LLMs show that, after fine-tuning, they markedly exceed the performance of much larger zero-shot models.<n>At the same time, the system still faces challenges in fully adhering to user instructions, underscoring the need for models with a better understanding of user-defined privacy preferences.
  arXiv  Detail & Related papers  (2025-07-07T18:22:55Z)
• MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation [54.410825977390274]
  Existing benchmarks to evaluate contextual privacy in LLM-agents primarily assess single-turn, low-complexity tasks.<n>We first present a benchmark - MAGPIE comprising 158 real-life high-stakes scenarios across 15 domains.<n>We then evaluate the current state-of-the-art LLMs on their understanding of contextually private data and their ability to collaborate without violating user privacy.
  arXiv  Detail & Related papers  (2025-06-25T18:04:25Z)
• The Eye of Sherlock Holmes: Uncovering User Private Attribute Profiling via Vision-Language Model Agentic Framework [28.25933078258213]
  A new privacy risk is associated with the ability to infer sensitive attributes from personal images.<n>This threat is particularly severe given that modern apps can easily access users' photo albums.<n>In this work, we construct PAPI, the largest dataset for studying private attribute profiling in personal images.<n>We also propose HolmesEye, a hybrid agentic framework that combines VLMs and LLMs to enhance privacy inference.
  arXiv  Detail & Related papers  (2025-05-25T13:22:10Z)
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action [54.11479432110771]
  PrivacyLens is a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories. We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds. State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions.
  arXiv  Detail & Related papers  (2024-08-29T17:58:38Z)
• Privacy-Aware Visual Language Models [16.272314073324626]
  This paper aims to advance our understanding of how Visual Language Models handle privacy-sensitive information. We introduce a new benchmark PrivBench, which contains images from 8 sensitive categories such as passports, or fingerprints. We evaluate 10 state-of-the-art VLMs on this benchmark and observe a generally limited understanding of privacy, highlighting a significant area for model improvement.
  arXiv  Detail & Related papers  (2024-05-27T17:59:25Z)
• Explaining models relating objects and privacy [33.78605193864911]
  We evaluate privacy models that use objects extracted from an image to determine why the image is predicted as private. We show that the presence of the person category and its cardinality is the main factor for the privacy decision.
  arXiv  Detail & Related papers  (2024-05-02T18:06:48Z)
• Private Attribute Inference from Images with Vision-Language Models [2.9373912230684565]
  Vision-language models (VLMs) are capable of understanding both images and text. We evaluate 7 state-of-the-art VLMs, finding that they can infer various personal attributes at up to 77.6% accuracy. We observe that accuracy scales with the general capabilities of the models, implying that future models can be misused as stronger inferential adversaries.
  arXiv  Detail & Related papers  (2024-04-16T14:42:49Z)
• LDP-Feat: Image Features with Local Differential Privacy [10.306943706927006]
  We propose two novel inversion attacks to show that it is possible to recover the original image features from embeddings. We propose the first method to privatize image features via local differential privacy, which, unlike prior approaches, provides a guaranteed bound for privacy leakage.
  arXiv  Detail & Related papers  (2023-08-22T06:28:55Z)
• ConfounderGAN: Protecting Image Data Privacy with Causal Confounder [85.6757153033139]
  We propose ConfounderGAN, a generative adversarial network (GAN) that can make personal image data unlearnable to protect the data privacy of its owners. Experiments are conducted in six image classification datasets, consisting of three natural object datasets and three medical datasets.
  arXiv  Detail & Related papers  (2022-12-04T08:49:14Z)
• Mixed Differential Privacy in Computer Vision [133.68363478737058]
  AdaMix is an adaptive differentially private algorithm for training deep neural network classifiers using both private and public image data. A few-shot or even zero-shot learning baseline that ignores private data can outperform fine-tuning on a large private dataset.
  arXiv  Detail & Related papers  (2022-03-22T06:15:43Z)
• Personalized Image Semantic Segmentation [58.980245748434]
  We generate more accurate segmentation results on unlabeled personalized images by investigating the data's personalized traits. We propose a baseline method that incorporates the inter-image context when segmenting certain images. The code and the PIS dataset will be made publicly available.
  arXiv  Detail & Related papers  (2021-07-24T04:03:11Z)
• Privacy-Preserving Image Features via Adversarial Affine Subspace Embeddings [72.68801373979943]
  Many computer vision systems require users to upload image features to the cloud for processing and storage. We propose a new privacy-preserving feature representation. Compared to the original features, our approach makes it significantly more difficult for an adversary to recover private information.
  arXiv  Detail & Related papers  (2020-06-11T17:29:48Z)
• TIPRDC: Task-Independent Privacy-Respecting Data Crowdsourcing Framework for Deep Learning with Anonymized Intermediate Representations [49.20701800683092]
  We present TIPRDC, a task-independent privacy-respecting data crowdsourcing framework with anonymized intermediate representation. The goal of this framework is to learn a feature extractor that can hide the privacy information from the intermediate representations; while maximally retaining the original information embedded in the raw data for the data collector to accomplish unknown learning tasks.
  arXiv  Detail & Related papers  (2020-05-23T06:21:26Z)
• InfoScrub: Towards Attribute Privacy by Targeted Obfuscation [77.49428268918703]
  We study techniques that allow individuals to limit the private information leaked in visual data. We tackle this problem in a novel image obfuscation framework. We find our approach generates obfuscated images faithful to the original input images, and additionally increase uncertainty by 6.2$times$ (or up to 0.85 bits) over the non-obfuscated counterparts.
  arXiv  Detail & Related papers  (2020-05-20T19:48:04Z)
• PrivEdge: From Local to Distributed Private Training and Prediction [43.02041269239928]
  PrivEdge is a technique for privacy-preserving Machine Learning (ML) PrivEdge safeguards the privacy of users who provide their data for training, as well as users who use the prediction service. We show that PrivEdge has high precision and recall in preserving privacy, as well as in distinguishing between private and non-private images.
  arXiv  Detail & Related papers  (2020-04-12T09:26:12Z)
• Privacy-Preserving Image Classification in the Local Setting [17.375582978294105]
  Local Differential Privacy (LDP) brings us a promising solution, which allows the data owners to randomly perturb their input to provide the plausible deniability of the data before releasing. In this paper, we consider a two-party image classification problem, in which data owners hold the image and the untrustworthy data user would like to fit a machine learning model with these images as input. We propose a supervised image feature extractor, DCAConv, which produces an image representation with scalable domain size.
  arXiv  Detail & Related papers  (2020-02-09T01:25:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.