PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say

• URL: http://arxiv.org/abs/2606.00152v1
• Date: Fri, 29 May 2026 04:55:12 GMT
• Title: PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say
• Abstract summary: LLM-based agents are rapidly advancing, autonomously invoking external tools to complete tasks for users.<n> Existing privacy benchmarks audit what the agent's response or outgoing actions disclose, but overlook the acquisition stage where data first enters the agent's context.<n>We introduce emphPrivacyPeek, a benchmark for evaluating acquisition-stage privacy leakage of LLM-based agents.
• Score: 48.84133320567554
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLM-based agents are rapidly advancing, autonomously invoking external tools to complete multi-step tasks for users. However, agents often acquire more sensitive information than the task requires. Existing privacy benchmarks audit what the agent's response or outgoing actions disclose, but overlook the acquisition stage where data first enters the agent's context. The over-acquired information is then one careless action or one attack away from an outright leak. To assess its prevalence, we introduce \emph{PrivacyPeek}, a benchmark for evaluating acquisition-stage privacy leakage of LLM-based agents, with $1{,}182$ cases across $7$ acquisition behaviours and $16$ application domains. Specifically, \emph{Acquisition Inspection} examines the agent's tool-call trajectory, both the tools it invokes and the data it receives, to detect when it acquires sensitive information beyond the task scope. \emph{Probe Elicitation} then issues a follow-up probe and measures how readily an attacker could elicit sensitive information the agent acquired but did not disclose. Our experiments on 10 LLM-based agents across 4 model families show that the unnecessary acquisition of sensitive information is widespread. In addition, we observe a correlation between the task-completion capability and acquisition-stage leakage. Prompt-level defences reduce only a small fraction of acquisition-stage leakage, leaving the majority unmitigated. These results make auditing acquisition-stage privacy both urgent and necessary. Our dataset and code are available at https://github.com/Xuan269/PrivacyPeek-Resource.

Related papers

• Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agents [1.0742675209112622]
  Large language model agents increasingly query databases, search document collections, call external APIs, remember past interactions, and act on a user's behalf.<n>As they move from answering questions to operating over sensitive data, privacy becomes harder to enforce.<n>We survey the privacy of LLM agents from a data-centric view, organizing the field around data an agent touches rather than by attack type.
  arXiv  Detail & Related papers  (2026-06-25T05:44:18Z)
• OCELOT: Inference-Leakage Budgets for Privacy-Preserving LLM Agents [32.5466552510287]
  Leakage is cumulative, as individually innocuous releases accumulate across honest-but-curious or colluding sinks.<n>We present OCELOT, a runtime mediator that budgets how much an adversary's belief about a secret may improve.<n>OCELOT attains significantly lower leakage at higher task utility, resists adaptive injection, jailbreak, cumulative inference, and sink collusion.
  arXiv  Detail & Related papers  (2026-06-10T17:13:35Z)
• MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agents [76.60965179634664]
  Deep research agents increasingly combine private local documents with external tools like web retrieval, creating a privacy risk.<n>We introduce MosaicLeaks, a benchmark of 1,001 deep research tasks that chain private enterprise documents and a public web corpus.<n>We find that models across families and sizes frequently leak at all three levels, that zero-shot privacy prompting reduces but does not eliminate leakage.
  arXiv  Detail & Related papers  (2026-05-29T01:44:35Z)
• Are Your Agents Upward Deceivers? [73.1073084327614]
  Large Language Model (LLM)-based agents are increasingly used as autonomous subordinates that carry out tasks for users.<n>This raises the question of whether they may also engage in deception, similar to how individuals in human organizations lie to superiors to create a good image or avoid punishment.<n>We observe and define agentic upward deception, a phenomenon in which an agent facing environmental constraints conceals its failure and performs actions that were not requested without reporting.
  arXiv  Detail & Related papers  (2025-12-04T14:47:05Z)
• 1-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning [18.751008976082655]
  We introduce a multi-agent framework that decomposes privacy reasoning into specialized subtasks (extraction, classification)<n>We conduct a systematic ablation over information-flow topologies, revealing when and why upstream detection mistakes cascade into downstream leakage.
  arXiv  Detail & Related papers  (2025-08-11T06:34:09Z)
• MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation [54.410825977390274]
  Existing benchmarks to evaluate contextual privacy in LLM-agents primarily assess single-turn, low-complexity tasks.<n>We first present a benchmark - MAGPIE comprising 158 real-life high-stakes scenarios across 15 domains.<n>We then evaluate the current state-of-the-art LLMs on their understanding of contextually private data and their ability to collaborate without violating user privacy.
  arXiv  Detail & Related papers  (2025-06-25T18:04:25Z)
• Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution [7.2497315292753415]
  This paper examines how prompt injection can cause tool-calling agents to leak personal data observed during task execution.<n>Using a fictitious banking agent, we develop data flow-based attacks and integrate them into AgentDojo, a recent benchmark for agentic security.
  arXiv  Detail & Related papers  (2025-06-01T15:48:06Z)
• Defeating Prompt Injections by Design [79.00910871948787]
  CaMeL is a robust defense that creates a protective system layer around the Large Language Models.<n>To operate, CaMeL explicitly extracts the control and data flows from the (trusted) query.<n>To further improve security, CaMeL uses a notion of a capability to prevent the exfiltration of private data over unauthorized data flows.
  arXiv  Detail & Related papers  (2025-03-24T15:54:10Z)
• AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents [66.29263282311258]
  We introduce a new benchmark AgentDAM that measures if AI web-navigation agents follow the privacy principle of data minimization''<n>Our benchmark simulates realistic web interaction scenarios end-to-end and is adaptable to all existing web navigation agents.
  arXiv  Detail & Related papers  (2025-03-12T19:30:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.