Do Vision-Language Models Respect Contextual Integrity in Location Disclosure?

• URL: http://arxiv.org/abs/2602.05023v1
• Date: Wed, 04 Feb 2026 20:24:14 GMT
• Title: Do Vision-Language Models Respect Contextual Integrity in Location Disclosure?
• Authors: Ruixin Yang, Ethan Mendes, Arthur Wang, James Hays, Sauvik Das, Wei Xu, Alan Ritter,
• Abstract summary: Vision-language models (VLMs) have demonstrated strong performance in image geolocation.<n>This poses a significant privacy risk as they can be exploited to infer sensitive locations from casually shared photos.<n>We introduce VLM-GEOPRIVACY, a benchmark that challenges VLMs to interpret latent social norms and contextual cues in real-world images.
• Score: 35.91273000038155
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Vision-language models (VLMs) have demonstrated strong performance in image geolocation, a capability further sharpened by frontier multimodal large reasoning models (MLRMs). This poses a significant privacy risk, as these widely accessible models can be exploited to infer sensitive locations from casually shared photos, often at street-level precision, potentially surpassing the level of detail the sharer consented or intended to disclose. While recent work has proposed applying a blanket restriction on geolocation disclosure to combat this risk, these measures fail to distinguish valid geolocation uses from malicious behavior. Instead, VLMs should maintain contextual integrity by reasoning about elements within an image to determine the appropriate level of information disclosure, balancing privacy and utility. To evaluate how well models respect contextual integrity, we introduce VLM-GEOPRIVACY, a benchmark that challenges VLMs to interpret latent social norms and contextual cues in real-world images and determine the appropriate level of location disclosure. Our evaluation of 14 leading VLMs shows that, despite their ability to precisely geolocate images, the models are poorly aligned with human privacy expectations. They often over-disclose in sensitive contexts and are vulnerable to prompt-based attacks. Our results call for new design principles in multimodal systems to incorporate context-conditioned privacy reasoning.

Related papers

• Beyond Pixels: Semantic-aware Typographic Attack for Geo-Privacy Protection [43.65944873827891]
  Large Visual Language Models (LVLMs) infer a social media user's geolocation directly from shared images, leading to unintended privacy leakage.<n> adversarial image perturbations provide a potential direction for geo-privacy protection, but require relatively strong distortions to be effective against LVLMs.<n>We identify deceptive attacks as a promising direction for protecting geo-privacy by adding text extension outside the visual content.
  arXiv  Detail & Related papers  (2025-11-16T12:27:59Z)
• Assessing the Geolocation Capabilities, Limitations and Societal Risks of Generative Vision-Language Models [11.444835352261002]
  Geo-localization is the task of identifying the location of an image using visual cues alone.<n>Vision-Language Models (VLMs) are increasingly demonstrating capabilities as accurate image geo-locators.<n>This brings significant privacy risks, including those related to stalking and surveillance.
  arXiv  Detail & Related papers  (2025-08-27T15:21:31Z)
• GeoShield: Safeguarding Geolocation Privacy from Vision-Language Models via Adversarial Perturbations [48.78781663571235]
  Vision-Language Models (VLMs) can infer users' locations from public shared images, posing a substantial risk to geoprivacy.<n>We propose GeoShield, a novel adversarial framework designed for robust geoprivacy protection in real-world scenarios.
  arXiv  Detail & Related papers  (2025-08-05T08:37:06Z)
• Evaluation of Geolocation Capabilities of Multimodal Large Language Models and Analysis of Associated Privacy Risks [9.003350058345442]
  MLLMs are capable of inferring the geographic location of images based solely on visual content.<n>This poses serious risks of privacy invasion, including doxxing, surveillance, and other security threats.<n>The most advanced visual models can successfully localize the origin of street-level imagery with up to $49%$ accuracy within a 1-kilometer radius.
  arXiv  Detail & Related papers  (2025-06-30T03:05:30Z)
• Transferable Adversarial Attacks on Black-Box Vision-Language Models [63.22532779621001]
  adversarial attacks can transfer from open-source to proprietary black-box models in text-only and vision-only contexts.<n>We show that attackers can craft perturbations to induce specific attacker-chosen interpretations of visual information.<n>We discover that universal perturbations -- modifications applicable to a wide set of images -- can consistently induce these misinterpretations.
  arXiv  Detail & Related papers  (2025-05-02T06:51:11Z)
• Doxing via the Lens: Revealing Location-related Privacy Leakage on Multi-modal Large Reasoning Models [37.18986847375693]
  Adversaries can infer sensitive geolocation information from user-generated images.<n>DoxBench is a curated dataset of 500 real-world images reflecting diverse privacy scenarios.<n>Our findings highlight the urgent need to reassess inference-time privacy risks in MLRMs.
  arXiv  Detail & Related papers  (2025-04-27T22:26:45Z)
• REVAL: A Comprehension Evaluation on Reliability and Values of Large Vision-Language Models [59.445672459851274]
  REVAL is a comprehensive benchmark designed to evaluate the textbfREliability and textbfVALue of Large Vision-Language Models.<n>REVAL encompasses over 144K image-text Visual Question Answering (VQA) samples, structured into two primary sections: Reliability and Values.<n>We evaluate 26 models, including mainstream open-source LVLMs and prominent closed-source models like GPT-4o and Gemini-1.5-Pro.
  arXiv  Detail & Related papers  (2025-03-20T07:54:35Z)
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action [54.11479432110771]
  PrivacyLens is a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories.<n>We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds.<n>State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions.
  arXiv  Detail & Related papers  (2024-08-29T17:58:38Z)
• Private Attribute Inference from Images with Vision-Language Models [2.9373912230684565]
  Vision-language models (VLMs) are capable of understanding both images and text. We evaluate 7 state-of-the-art VLMs, finding that they can infer various personal attributes at up to 77.6% accuracy. We observe that accuracy scales with the general capabilities of the models, implying that future models can be misused as stronger inferential adversaries.
  arXiv  Detail & Related papers  (2024-04-16T14:42:49Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• Benchmarking Zero-Shot Recognition with Vision-Language Models: Challenges on Granularity and Specificity [45.86789047206224]
  This paper presents novel benchmarks for evaluating vision-language models (VLMs) in zero-shot recognition. Our benchmarks test VLMs' consistency in understanding concepts across semantic granularity levels and their response to varying text specificity. Findings show that VLMs favor moderately fine-grained concepts and struggle with specificity, often misjudging texts that differ from their training data.
  arXiv  Detail & Related papers  (2023-06-28T09:29:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.