Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmark

• URL: http://arxiv.org/abs/2510.02356v2
• Date: Mon, 13 Oct 2025 17:24:22 GMT
• Title: Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmark
• Abstract summary: We introduce EAPrivacy, a comprehensive evaluation benchmark designed to quantify the physical-world privacy awareness of embodied agents.<n>EAPrivacy utilizes procedurally generated scenarios across four tiers to test an agent's ability to handle sensitive objects, adapt to changing environments, balance task execution with privacy constraints, and resolve conflicts with social norms.
• Score: 10.645183619391135
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The deployment of Large Language Models (LLMs) in embodied agents creates an urgent need to measure their privacy awareness in the physical world. Existing evaluation methods, however, are confined to natural language based scenarios. To bridge this gap, we introduce EAPrivacy, a comprehensive evaluation benchmark designed to quantify the physical-world privacy awareness of LLM-powered agents. EAPrivacy utilizes procedurally generated scenarios across four tiers to test an agent's ability to handle sensitive objects, adapt to changing environments, balance task execution with privacy constraints, and resolve conflicts with social norms. Our measurements reveal a critical deficit in current models. The top-performing model, Gemini 2.5 Pro, achieved only 59\% accuracy in scenarios involving changing physical environments. Furthermore, when a task was accompanied by a privacy request, models prioritized completion over the constraint in up to 86\% of cases. In high-stakes situations pitting privacy against critical social norms, leading models like GPT-4o and Claude-3.5-haiku disregarded the social norm over 15\% of the time. These findings, demonstrated by our benchmark, underscore a fundamental misalignment in LLMs regarding physically grounded privacy and establish the need for more robust, physically-aware alignment. Codes and datasets will be available at https://github.com/Graph-COM/EAPrivacy.

Related papers

• Seeing Without Exposing: Adaptive Privacy Control for Open-World, Context-Hungry MLLMs [61.80513991207956]
  We propose Anchored Privacy Drifting (APD) to tackle privacy challenges in MLLMs.<n>APD drifts privacy-sensitive elements toward semantically equivalent alternatives while anchoring contextual cues to the source image.<n>We show that our method achieves balanced improvements in both privacy sanitization and content retention.
  arXiv  Detail & Related papers  (2026-06-05T11:41:39Z)
• How Far Are VLMs from Privacy Awareness in the Physical World? An Empirical Study [7.537653216205245]
  Vision-Language Models (VLMs) are increasingly deployed as autonomous cognitive cores for embodied assistants.<n>We present ImmersedPrivacy, an interactive audio-visual evaluation framework that simulates realistic physical environments.<n>Our evaluation of 12 state-of-the-art models reveals consistent deficits.
  arXiv  Detail & Related papers  (2026-05-06T18:10:51Z)
• Unintended Memorization of Sensitive Information in Fine-Tuned Language Models [24.228889351240838]
  Fine-tuning Large Language Models (LLMs) on sensitive datasets carries a substantial risk of unintended memorization and leakage of Personally Identifiable Information (PII)<n>We design controlled extraction probes to quantify unintended PII memorization and study how factors such as language, PII frequency, task type, and model size influence memorization behavior.
  arXiv  Detail & Related papers  (2026-01-24T15:08:45Z)
• Privacy Collapse: Benign Fine-Tuning Can Break Contextual Privacy in Language Models [47.866853046761044]
  We find that diverse, subtle patterns in training data can degrade contextual privacy.<n>Fine-tuned models lose their ability to reason about contextual privacy norms.<n>Our results reveal a critical gap in current safety evaluations.
  arXiv  Detail & Related papers  (2026-01-21T17:53:06Z)
• FusionDP: Foundation Model-Assisted Differentially Private Learning for Partially Sensitive Features [17.945111987608865]
  In practical scenarios, privacy protection may be required for only a subset of features.<n>Traditional DP-SGD enforces privacy protection on all features in one sample, leading to excessive noise injection and significant utility degradation.<n>We propose FusionDP, a two-step framework that enhances model utility under feature-level differential privacy.
  arXiv  Detail & Related papers  (2025-11-05T19:13:10Z)
• Adaptive Backtracking for Privacy Protection in Large Language Models [22.340767807000212]
  This paper introduces a novel objective: enterprise-oriented privacy concerns.<n>Existing methods such as data sanitization severely degrade model performance.<n>We propose ABack, a training-free mechanism that leverages a Hidden State Model.
  arXiv  Detail & Related papers  (2025-08-08T07:29:33Z)
• MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation [54.410825977390274]
  Existing benchmarks to evaluate contextual privacy in LLM-agents primarily assess single-turn, low-complexity tasks.<n>We first present a benchmark - MAGPIE comprising 158 real-life high-stakes scenarios across 15 domains.<n>We then evaluate the current state-of-the-art LLMs on their understanding of contextually private data and their ability to collaborate without violating user privacy.
  arXiv  Detail & Related papers  (2025-06-25T18:04:25Z)
• Personalized Language Models via Privacy-Preserving Evolutionary Model Merging [53.97323896430374]
  Personalization in language models aims to tailor model behavior to individual users or user groups.<n>We propose Privacy-Preserving Model Merging via Evolutionary Algorithms (PriME)<n>PriME employs gradient-free methods to directly optimize utility while reducing privacy risks.<n>Experiments on the LaMP benchmark show that PriME consistently outperforms a range of baselines, achieving up to a 45% improvement in task performance.
  arXiv  Detail & Related papers  (2025-03-23T09:46:07Z)
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action [54.11479432110771]
  PrivacyLens is a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories.<n>We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds.<n>State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions.
  arXiv  Detail & Related papers  (2024-08-29T17:58:38Z)
• Representation Magnitude has a Liability to Privacy Vulnerability [3.301728339780329]
  We propose a plug-in model-level solution to mitigate membership privacy leakage. Our approach ameliorates models' privacy vulnerability while maintaining generalizability.
  arXiv  Detail & Related papers  (2024-07-23T04:13:52Z)
• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
  differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit. We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
  arXiv  Detail & Related papers  (2024-06-20T13:54:32Z)
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory [82.7042006247124]
  We show that even the most capable AI models reveal private information in contexts that humans would not, 39% and 57% of the time, respectively. Our work underscores the immediate need to explore novel inference-time privacy-preserving approaches, based on reasoning and theory of mind.
  arXiv  Detail & Related papers  (2023-10-27T04:15:30Z)
• Can Foundation Models Help Us Achieve Perfect Secrecy? [11.073539163281524]
  A key promise of machine learning is the ability to assist users with personal tasks. A gold standard privacy-preserving system will satisfy perfect secrecy. However, privacy and quality appear to be in tension in existing systems for personal tasks.
  arXiv  Detail & Related papers  (2022-05-27T02:32:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.