Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems

• URL: http://arxiv.org/abs/2605.27766v1
• Date: Tue, 26 May 2026 23:32:25 GMT
• Title: Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems
• Abstract summary: We introduce a Moltbook-style simulation platform where thousands of LLM agents interact across communities over a simulated month.<n>We find that shifting from single turn to multi turn social evaluation amplifies privacy violations.<n>Agents 8 times more likely to disclose sensitive information after observing a peer do so.
• Score: 4.480003364414813
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM safety evaluations predominantly test models in isolation, yet deployed AI agents increasingly operate within persistent social environments alongside other agents. We introduce a Moltbook-style simulation platform where thousands of LLM agents interact across communities over a simulated month, and use it to evaluate privacy as a downstream safety concern under varying degrees of social pressure. We find that shifting from single turn to multi turn social evaluation amplifies privacy violations (CIMemories 19.95% to Ours 45.30% across OpenAI models), that leakage is socially contagious, with agents 8 times more likely to disclose sensitive information after observing a peer do so, and that explicit privacy instructions reduce but do not eliminate this effect, leaving leakage rates above 37.8% even with safeguards. Our findings suggest that static chat based safety benchmarks systematically underestimate risks in agentic deployment, and that social context alone is sufficient to elicit sensitive disclosures that single turn evaluations would never surface.

Related papers

• MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agents [76.60965179634664]
  Deep research agents increasingly combine private local documents with external tools like web retrieval, creating a privacy risk.<n>We introduce MosaicLeaks, a benchmark of 1,001 deep research tasks that chain private enterprise documents and a public web corpus.<n>We find that models across families and sizes frequently leak at all three levels, that zero-shot privacy prompting reduces but does not eliminate leakage.
  arXiv  Detail & Related papers  (2026-05-29T01:44:35Z)
• AgentSocialBench: Evaluating Privacy Risks in Human-Centered Agentic Social Networks [2.291770711277359]
  Privacy in agentic social networks is fundamentally harder than in single-agent settings.<n>cross-domain and cross-user coordination creates persistent leakage pressure.<n>Privacy instructions that teach agents how to abstract sensitive information paradoxically cause them to discuss it more.
  arXiv  Detail & Related papers  (2026-04-01T23:43:23Z)
• ConVerse: Benchmarking Contextual Safety in Agent-to-Agent Conversations [11.177126931962443]
  ConVerse is a benchmark for evaluating privacy and security risks in agent-agent interactions.<n>It spans three practical domains with 12 user personas and over 864 contextually grounded attacks.<n>By unifying privacy and security within interactive multi-agent contexts, ConVerse reframes safety as an emergent property of communication.
  arXiv  Detail & Related papers  (2025-11-07T15:49:49Z)
• Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents [40.39717403627143]
  We present PrivacyChecker, a model-agnostic, contextual integrity based mitigation approach.<n>We also introduce PrivacyLens-Live, transforming static benchmarks into dynamic MCP and A2A environments.<n>Our data and code will be made available at https://aka.ms/privacy_in_action.
  arXiv  Detail & Related papers  (2025-09-22T08:19:06Z)
• The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration [72.33801123508145]
  Large language models (LLMs) are integral to multi-agent systems.<n>Privacy risks emerge that extend beyond memorization, direct inference, or single-turn evaluations.<n>In particular, seemingly innocuous responses, when composed across interactions, can cumulatively enable adversaries to recover sensitive information.
  arXiv  Detail & Related papers  (2025-09-16T16:57:25Z)
• 1-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning [18.751008976082655]
  We introduce a multi-agent framework that decomposes privacy reasoning into specialized subtasks (extraction, classification)<n>We conduct a systematic ablation over information-flow topologies, revealing when and why upstream detection mistakes cascade into downstream leakage.
  arXiv  Detail & Related papers  (2025-08-11T06:34:09Z)
• MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation [54.410825977390274]
  Existing benchmarks to evaluate contextual privacy in LLM-agents primarily assess single-turn, low-complexity tasks.<n>We first present a benchmark - MAGPIE comprising 158 real-life high-stakes scenarios across 15 domains.<n>We then evaluate the current state-of-the-art LLMs on their understanding of contextually private data and their ability to collaborate without violating user privacy.
  arXiv  Detail & Related papers  (2025-06-25T18:04:25Z)
• Agent-SafetyBench: Evaluating the Safety of LLM Agents [72.92604341646691]
  We introduce Agent-SafetyBench, a benchmark designed to evaluate the safety of large language models (LLMs)<n>Agent-SafetyBench encompasses 349 interaction environments and 2,000 test cases, evaluating 8 categories of safety risks and covering 10 common failure modes frequently encountered in unsafe interactions.<n>Our evaluation of 16 popular LLM agents reveals a concerning result: none of the agents achieves a safety score above 60%.
  arXiv  Detail & Related papers  (2024-12-19T02:35:15Z)
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action [54.11479432110771]
  PrivacyLens is a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories.<n>We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds.<n>State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions.
  arXiv  Detail & Related papers  (2024-08-29T17:58:38Z)
• The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies [58.94148083602662]
  Large Language Models (LLMs) agents have evolved to perform complex tasks.<n>The widespread applications of LLM agents demonstrate their significant commercial value.<n>However, they also expose security and privacy vulnerabilities.<n>This survey aims to provide a comprehensive overview of the newly emerged privacy and security issues faced by LLM agents.
  arXiv  Detail & Related papers  (2024-07-28T00:26:24Z)
• R-Judge: Benchmarking Safety Risk Awareness for LLM Agents [28.0550468465181]
  Large language models (LLMs) have exhibited great potential in autonomously completing tasks across real-world applications. This work addresses the imperative need for benchmarking the behavioral safety of LLM agents within diverse environments. We introduce R-Judge, a benchmark crafted to evaluate the proficiency of LLMs in judging and identifying safety risks given agent interaction records.
  arXiv  Detail & Related papers  (2024-01-18T14:40:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.