PANOPTICON: A PII-Based Assemblage of Naturalistic Output Tokens for Investigating Privacy Leakage Within LLM Context Window

• URL: http://arxiv.org/abs/2607.22695v1
• Date: Fri, 17 Jul 2026 17:23:39 GMT
• Title: PANOPTICON: A PII-Based Assemblage of Naturalistic Output Tokens for Investigating Privacy Leakage Within LLM Context Window
• Abstract summary: Large Language Models (LLMs) are capable of generalizing human language for the completion of never-before-seen tasks.<n>These tasks often require the insertion of Personally Identifiable Information (PII), strings of information that uniquely identify some individual, raising privacy concerns.<n>We introduce the PANOPTICON pipeline and dataset, containing PII spans from 9,674 publicly available synthetic user profiles.
• Score: 0.49703640686206074
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) are capable of generalizing human language for the completion of never-before-seen tasks, leading to widespread deployment. While this automation provides clear utility, completing these tasks often requires the insertion of Personally Identifiable Information (PII), strings of information that uniquely identify some individual, raising privacy concerns. However, ethics has prevented the curation of a public, authentic dataset of PII. Without an appropriate dataset, it is difficult to quantify privacy risks. Thus, we introduce the PANOPTICON pipeline and dataset. The dataset, generated by Meta's Llama-3.1-8B-Instruct model, contains 67, 718 prompts, intended for the models context window, containing PII spans derived from 9,674 publicly available synthetic user profiles. We measure lexical diversity and S-BERT diversity of the created dataset to evaluate realism. Finally, we present a case study showcasing the utility of PANOPTICON data for understanding Prompt Inversion Attacks (PIAs). PANOPTICON thus emerges as the first benchmark dataset for studying PIAs over private corpora, providing a foundation for future LLM privacy research.

Related papers

• Beyond Direct Identifiers: Probabilistic Privacy Risk Estimation for Privacy-Conscious LLM Query Delegation [20.31515213689976]
  Privacy risk does not stem solely from explicit identifiers but also PII-free self-disclosures.<n>We propose k-anonymity as a useful auxiliary metric for tackling Privacy-Conscious Delegation.
  arXiv  Detail & Related papers  (2026-08-10T05:38:28Z)
• IDP-Bench: Benchmarking ability of LLMs to protect personal information in interdependent privacy contexts [15.026169063578697]
  Large language models (LLMs) are becoming widely deployed as personal AI assistants with access to sensitive user data.<n>Prior work focuses mainly on individual-level risks, overlooking textbfinterdependent privacy (IDP)--where one person's data may be revealed by others without their knowledge or consent.<n>We introduce textbfIDP-Bench: the first LLM benchmark for IDP scenarios, grounded in the Contextual Integrity (CI) framework.
  arXiv  Detail & Related papers  (2026-06-06T07:59:08Z)
• Profiling for Pennies: Unveiling the Privacy Iceberg of LLM Agents [52.02752011119451]
  We investigate privacy perception in the real world through the lens of human awareness.<n>We propose the PrivacyIceberg, which categorizes real-world human privacy risks into three tiers.
  arXiv  Detail & Related papers  (2026-05-07T13:21:44Z)
• CAPID: Context-Aware PII Detection for Question-Answering Systems [2.538582648751871]
  We propose CAPID, a practical approach that fine-tunes a locally owned small language model (SLM) that filters sensitive information before it is passed to LLMs for QA.<n>Existing datasets do not capture the context-dependent relevance of PII needed to train such a model effectively.<n>Our experiments show that relevance-aware PII detection with a fine-tuned SLM substantially outperforms existing baselines in span, relevance and type accuracy.
  arXiv  Detail & Related papers  (2026-02-10T18:41:31Z)
• How to DP-fy Your Data: A Practical Guide to Generating Synthetic Data With Differential Privacy [52.00934156883483]
  Differential Privacy (DP) is a framework for reasoning about and limiting information leakage.<n>Differentially Private Synthetic data refers to synthetic data that preserves the overall trends of source data.
  arXiv  Detail & Related papers  (2025-12-02T21:14:39Z)
• Safe-LLaVA: A Privacy-Preserving Vision-Language Dataset and Benchmark for Biometric Safety [43.17090130312271]
  Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in vision-language tasks.<n>These models often infer and reveal sensitive biometric attributes such as race, gender, age, body weight, and eye color.<n>Despite increasing awareness, no publicly available dataset or benchmark exists to comprehensively evaluate or mitigate biometric leakage in MLLMs.
  arXiv  Detail & Related papers  (2025-08-29T18:54:57Z)
• MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation [54.410825977390274]
  Existing benchmarks to evaluate contextual privacy in LLM-agents primarily assess single-turn, low-complexity tasks.<n>We first present a benchmark - MAGPIE comprising 158 real-life high-stakes scenarios across 15 domains.<n>We then evaluate the current state-of-the-art LLMs on their understanding of contextually private data and their ability to collaborate without violating user privacy.
  arXiv  Detail & Related papers  (2025-06-25T18:04:25Z)
• PANORAMA: A synthetic PII-laced dataset for studying sensitive data memorization in LLMs [0.0]
  memorization of sensitive and personally identifiable information poses growing privacy risks.<n>Existing efforts to study sensitive and PII data memorization and develop mitigation strategies are hampered by the absence of realistic datasets.<n>We introduce PANORAMA - Profile-based Assemblage for Naturalistic Online Representation and Attribute Memorization Analysis.
  arXiv  Detail & Related papers  (2025-05-18T05:27:35Z)
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action [54.11479432110771]
  PrivacyLens is a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories.<n>We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds.<n>State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions.
  arXiv  Detail & Related papers  (2024-08-29T17:58:38Z)
• Robust Utility-Preserving Text Anonymization Based on Large Language Models [80.5266278002083]
  Anonymizing text that contains sensitive information is crucial for a wide range of applications.<n>Existing techniques face the emerging challenges of the re-identification ability of large language models.<n>We propose a framework composed of three key components: a privacy evaluator, a utility evaluator, and an optimization component.
  arXiv  Detail & Related papers  (2024-07-16T14:28:56Z)
• DiscoveryBench: Towards Data-Driven Discovery with Large Language Models [50.36636396660163]
  We present DiscoveryBench, the first comprehensive benchmark that formalizes the multi-step process of data-driven discovery. Our benchmark contains 264 tasks collected across 6 diverse domains, such as sociology and engineering. Our benchmark, thus, illustrates the challenges in autonomous data-driven discovery and serves as a valuable resource for the community to make progress.
  arXiv  Detail & Related papers  (2024-07-01T18:58:22Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• ProPILE: Probing Privacy Leakage in Large Language Models [38.92840523665835]
  Large language models (LLMs) are often trained on vast quantities of web-collected data, which may inadvertently include sensitive personal data. This paper presents ProPILE, a novel probing tool designed to empower data subjects, or the owners of the PII, with awareness of potential PII leakage.
  arXiv  Detail & Related papers  (2023-07-04T18:53:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.