ProPILE: Probing Privacy Leakage in Large Language Models
- URL: http://arxiv.org/abs/2307.01881v1
- Date: Tue, 4 Jul 2023 18:53:47 GMT
- Title: ProPILE: Probing Privacy Leakage in Large Language Models
- Authors: Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, Seong
Joon Oh
- Abstract summary: Large language models (LLMs) are often trained on vast quantities of web-collected data, which may inadvertently include sensitive personal data.
This paper presents ProPILE, a novel probing tool designed to empower data subjects, or the owners of the PII, with awareness of potential PII leakage.
- Score: 38.92840523665835
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: The rapid advancement and widespread use of large language models (LLMs) have
raised significant concerns regarding the potential leakage of personally
identifiable information (PII). These models are often trained on vast
quantities of web-collected data, which may inadvertently include sensitive
personal data. This paper presents ProPILE, a novel probing tool designed to
empower data subjects, or the owners of the PII, with awareness of potential
PII leakage in LLM-based services. ProPILE lets data subjects formulate prompts
based on their own PII to evaluate the level of privacy intrusion in LLMs. We
demonstrate its application on the OPT-1.3B model trained on the publicly
available Pile dataset. We show how hypothetical data subjects may assess the
likelihood of their PII being included in the Pile dataset being revealed.
ProPILE can also be leveraged by LLM service providers to effectively evaluate
their own levels of PII leakage with more powerful prompts specifically tuned
for their in-house models. This tool represents a pioneering step towards
empowering the data subjects for their awareness and control over their own
data on the web.
Related papers
- Robust Utility-Preserving Text Anonymization Based on Large Language Models [80.5266278002083]
Text anonymization is crucial for sharing sensitive data while maintaining privacy.
Existing techniques face the emerging challenges of re-identification attack ability of Large Language Models.
This paper proposes a framework composed of three LLM-based components -- a privacy evaluator, a utility evaluator, and an optimization component.
arXiv Detail & Related papers (2024-07-16T14:28:56Z) - Enhancing Data Privacy in Large Language Models through Private Association Editing [1.078439500019266]
Large Language Models (LLMs) are powerful tools with extensive applications, but their tendency to memorize private information raises significant concerns.
This paper introduces Private Association Editing (PAE), a novel defense approach for private data leakage.
arXiv Detail & Related papers (2024-06-26T10:08:47Z) - The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks [19.364127374679253]
We propose a novel attack, Janus, which exploits the fine-tuning interface to recover forgotten PIIs from the pre-training data in language models.
Our experiment results show that Janus amplifies the privacy risks by over 10 times in comparison with the baseline.
Our analysis validates that existing fine-tuning APIs provided by OpenAI and Azure AI Studio are susceptible to our Janus attack.
arXiv Detail & Related papers (2023-10-24T02:48:19Z) - Large Language Models Can Be Good Privacy Protection Learners [53.07930843882592]
We introduce Privacy Protection Language Models (PPLM), a novel paradigm for fine-tuning language models.
Our work offers a theoretical analysis for model design and delves into various techniques such as corpus curation, penalty-based unlikelihood in training loss, and instruction-based tuning.
In particular, instruction tuning with both positive and negative examples, stands out as a promising method, effectively protecting private data while enhancing the model's knowledge.
arXiv Detail & Related papers (2023-10-03T22:37:01Z) - WASA: WAtermark-based Source Attribution for Large Language
Model-Generated Data [60.759755177369364]
Large language models (LLMs) generate synthetic texts with embedded watermarks that contain information about their source(s)
We propose a WAtermarking for Source Attribution (WASA) framework that satisfies key properties due to our algorithmic designs.
Our framework achieves effective source attribution and data provenance.
arXiv Detail & Related papers (2023-10-01T12:02:57Z) - Quantifying Association Capabilities of Large Language Models and Its
Implications on Privacy Leakage [28.385083741414213]
This paper delves into the association capabilities of language models, aiming to uncover the factors that influence their proficiency in associating information.
Our study reveals that as models scale up, their capacity to associate entities/information intensifies, particularly when target pairs demonstrate shorter co-occurrence distances or higher co-occurrence frequencies.
Despite the proportion of accurately predicted PII being relatively small, LLMs still demonstrate the capability to predict specific instances of email addresses and phone numbers when provided with appropriate prompts.
arXiv Detail & Related papers (2023-05-22T04:30:35Z) - DP2-Pub: Differentially Private High-Dimensional Data Publication with
Invariant Post Randomization [58.155151571362914]
We propose a differentially private high-dimensional data publication mechanism (DP2-Pub) that runs in two phases.
splitting attributes into several low-dimensional clusters with high intra-cluster cohesion and low inter-cluster coupling helps obtain a reasonable privacy budget.
We also extend our DP2-Pub mechanism to the scenario with a semi-honest server which satisfies local differential privacy.
arXiv Detail & Related papers (2022-08-24T17:52:43Z) - Automated PII Extraction from Social Media for Raising Privacy
Awareness: A Deep Transfer Learning Approach [6.806025738284367]
Internet users have been exposing an increasing amount of Personally Identifiable Information (PII) on social media.
In this study, we propose the Deep Transfer Learning for PII Extraction (DTL-PIIE) framework to address these two limitations.
Our framework can facilitate various applications, such as PII misuse prediction and privacy risk assessment.
arXiv Detail & Related papers (2021-11-11T19:32:05Z) - Survey: Leakage and Privacy at Inference Time [59.957056214792665]
Leakage of data from publicly available Machine Learning (ML) models is an area of growing significance.
We focus on inference-time leakage, as the most likely scenario for publicly available models.
We propose a taxonomy across involuntary and malevolent leakage, available defences, followed by the currently available assessment metrics and applications.
arXiv Detail & Related papers (2021-07-04T12:59:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.