ProPILE: Probing Privacy Leakage in Large Language Models

• URL: http://arxiv.org/abs/2307.01881v1
• Date: Tue, 4 Jul 2023 18:53:47 GMT
• Title: ProPILE: Probing Privacy Leakage in Large Language Models
• Authors: Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, Seong Joon Oh
• Abstract summary: Large language models (LLMs) are often trained on vast quantities of web-collected data, which may inadvertently include sensitive personal data. This paper presents ProPILE, a novel probing tool designed to empower data subjects, or the owners of the PII, with awareness of potential PII leakage.
• Score: 38.92840523665835
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The rapid advancement and widespread use of large language models (LLMs) have raised significant concerns regarding the potential leakage of personally identifiable information (PII). These models are often trained on vast quantities of web-collected data, which may inadvertently include sensitive personal data. This paper presents ProPILE, a novel probing tool designed to empower data subjects, or the owners of the PII, with awareness of potential PII leakage in LLM-based services. ProPILE lets data subjects formulate prompts based on their own PII to evaluate the level of privacy intrusion in LLMs. We demonstrate its application on the OPT-1.3B model trained on the publicly available Pile dataset. We show how hypothetical data subjects may assess the likelihood of their PII being included in the Pile dataset being revealed. ProPILE can also be leveraged by LLM service providers to effectively evaluate their own levels of PII leakage with more powerful prompts specifically tuned for their in-house models. This tool represents a pioneering step towards empowering the data subjects for their awareness and control over their own data on the web.

Related papers

• Augmenting Anonymized Data with AI: Exploring the Feasibility and Limitations of Large Language Models in Data Enrichment [3.459382629188014]
  Large Language Models (LLMs) have demonstrated advanced capabilities in both text generation and comprehension. Their application to data archives might facilitate the privatization of sensitive information about the data subjects. This data, if not safeguarded, may bring privacy risks in terms of both disclosure and identification.
  arXiv  Detail & Related papers  (2025-04-03T13:26:59Z)
• Information-Guided Identification of Training Data Imprint in (Proprietary) Large Language Models [52.439289085318634]
  We show how to identify training data known to proprietary large language models (LLMs) by using information-guided probes. Our work builds on a key observation: text passages with high surprisal are good search material for memorization probes.
  arXiv  Detail & Related papers  (2025-03-15T10:19:15Z)
• Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data [18.984529269623135]
  This study investigates whether fine-tuning with generated data truly enhances privacy or introduces additional privacy risks. We use the Pythia Model Suite and Open Pre-trained Transformer to measure privacy risks.
  arXiv  Detail & Related papers  (2024-09-12T10:14:12Z)
• LLM-PBE: Assessing Data Privacy in Large Language Models [111.58198436835036]
  Large Language Models (LLMs) have become integral to numerous domains, significantly advancing applications in data management, mining, and analysis. Despite the critical nature of this issue, there has been no existing literature to offer a comprehensive assessment of data privacy risks in LLMs. Our paper introduces LLM-PBE, a toolkit crafted specifically for the systematic evaluation of data privacy risks in LLMs.
  arXiv  Detail & Related papers  (2024-08-23T01:37:29Z)
• Evaluating Large Language Model based Personal Information Extraction and Countermeasures [63.91918057570824]
  Large language model (LLM) can be misused by attackers to accurately extract various personal information from personal profiles. LLM outperforms conventional methods at such extraction. prompt injection can mitigate such risk to a large extent and outperforms conventional countermeasures.
  arXiv  Detail & Related papers  (2024-08-14T04:49:30Z)
• Robust Utility-Preserving Text Anonymization Based on Large Language Models [80.5266278002083]
  Text anonymization is crucial for sharing sensitive data while maintaining privacy. Existing techniques face the emerging challenges of re-identification attack ability of Large Language Models. This paper proposes a framework composed of three LLM-based components -- a privacy evaluator, a utility evaluator, and an optimization component.
  arXiv  Detail & Related papers  (2024-07-16T14:28:56Z)
• The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks [19.364127374679253]
  We propose a novel attack, Janus, which exploits the fine-tuning interface to recover forgotten PIIs from the pre-training data in language models. Our experiment results show that Janus amplifies the privacy risks by over 10 times in comparison with the baseline. Our analysis validates that existing fine-tuning APIs provided by OpenAI and Azure AI Studio are susceptible to our Janus attack.
  arXiv  Detail & Related papers  (2023-10-24T02:48:19Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• Quantifying Association Capabilities of Large Language Models and Its Implications on Privacy Leakage [28.385083741414213]
  This paper delves into the association capabilities of language models, aiming to uncover the factors that influence their proficiency in associating information. Our study reveals that as models scale up, their capacity to associate entities/information intensifies, particularly when target pairs demonstrate shorter co-occurrence distances or higher co-occurrence frequencies. Despite the proportion of accurately predicted PII being relatively small, LLMs still demonstrate the capability to predict specific instances of email addresses and phone numbers when provided with appropriate prompts.
  arXiv  Detail & Related papers  (2023-05-22T04:30:35Z)
• Automated PII Extraction from Social Media for Raising Privacy Awareness: A Deep Transfer Learning Approach [6.806025738284367]
  Internet users have been exposing an increasing amount of Personally Identifiable Information (PII) on social media. In this study, we propose the Deep Transfer Learning for PII Extraction (DTL-PIIE) framework to address these two limitations. Our framework can facilitate various applications, such as PII misuse prediction and privacy risk assessment.
  arXiv  Detail & Related papers  (2021-11-11T19:32:05Z)
• Survey: Leakage and Privacy at Inference Time [59.957056214792665]
  Leakage of data from publicly available Machine Learning (ML) models is an area of growing significance. We focus on inference-time leakage, as the most likely scenario for publicly available models. We propose a taxonomy across involuntary and malevolent leakage, available defences, followed by the currently available assessment metrics and applications.
  arXiv  Detail & Related papers  (2021-07-04T12:59:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.