You Are What You Write: Preserving Privacy in the Era of Large Language Models

• URL: http://arxiv.org/abs/2204.09391v1
• Date: Wed, 20 Apr 2022 11:12:53 GMT
• Title: You Are What You Write: Preserving Privacy in the Era of Large Language Models
• Authors: Richard Plant, Valerio Giuffrida, Dimitra Gkatzia
• Abstract summary: We present an empirical investigation into the extent of the personal information encoded into pre-trained representations by a range of popular models. We show a positive correlation between the complexity of a model, the amount of data used in pre-training, and data leakage.
• Score: 2.3431670397288005
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Large scale adoption of large language models has introduced a new era of convenient knowledge transfer for a slew of natural language processing tasks. However, these models also run the risk of undermining user trust by exposing unwanted information about the data subjects, which may be extracted by a malicious party, e.g. through adversarial attacks. We present an empirical investigation into the extent of the personal information encoded into pre-trained representations by a range of popular models, and we show a positive correlation between the complexity of a model, the amount of data used in pre-training, and data leakage. In this paper, we present the first wide coverage evaluation and comparison of some of the most popular privacy-preserving algorithms, on a large, multi-lingual dataset on sentiment analysis annotated with demographic information (location, age and gender). The results show since larger and more complex models are more prone to leaking private information, use of privacy-preserving methods is highly desirable. We also find that highly privacy-preserving technologies like differential privacy (DP) can have serious model utility effects, which can be ameliorated using hybrid or metric-DP techniques.

Related papers

• Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakage [12.892449128678516]
  Fine-tuning language models on private data for downstream applications poses significant privacy risks. Several popular community platforms now offer convenient distribution of a large variety of pre-trained models. We introduce a novel poisoning technique that uses model-unlearning as an attack tool.
  arXiv  Detail & Related papers  (2024-08-30T15:35:09Z)
• Privacy-preserving datasets by capturing feature distributions with Conditional VAEs [0.11999555634662634]
  Conditional Variational Autoencoders (CVAEs) trained on feature vectors extracted from large pre-trained vision foundation models. Our method notably outperforms traditional approaches in both medical and natural image domains. Results underscore the potential of generative models to significantly impact deep learning applications in data-scarce and privacy-sensitive environments.
  arXiv  Detail & Related papers  (2024-08-01T15:26:24Z)
• Robust Utility-Preserving Text Anonymization Based on Large Language Models [80.5266278002083]
  Text anonymization is crucial for sharing sensitive data while maintaining privacy. Existing techniques face the emerging challenges of re-identification attack ability of Large Language Models. This paper proposes a framework composed of three LLM-based components -- a privacy evaluator, a utility evaluator, and an optimization component.
  arXiv  Detail & Related papers  (2024-07-16T14:28:56Z)
• Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models [112.48136829374741]
  In this paper, we unveil a new vulnerability: the privacy backdoor attack. When a victim fine-tunes a backdoored model, their training data will be leaked at a significantly higher rate than if they had fine-tuned a typical model. Our findings highlight a critical privacy concern within the machine learning community and call for a reevaluation of safety protocols in the use of open-source pre-trained models.
  arXiv  Detail & Related papers  (2024-04-01T16:50:54Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• A Unified View of Differentially Private Deep Generative Modeling [60.72161965018005]
  Data with privacy concerns comes with stringent regulations that frequently prohibited data access and data sharing. Overcoming these obstacles is key for technological progress in many real-world application scenarios that involve privacy sensitive data. Differentially private (DP) data publishing provides a compelling solution, where only a sanitized form of the data is publicly released.
  arXiv  Detail & Related papers  (2023-09-27T14:38:16Z)
• Recovering from Privacy-Preserving Masking with Large Language Models [14.828717714653779]
  We use large language models (LLMs) to suggest substitutes of masked tokens. We show that models trained on the obfuscation corpora are able to achieve comparable performance with the ones trained on the original data.
  arXiv  Detail & Related papers  (2023-09-12T16:39:41Z)
• Private Set Generation with Discriminative Information [63.851085173614]
  Differentially private data generation is a promising solution to the data privacy challenge. Existing private generative models are struggling with the utility of synthetic samples. We introduce a simple yet effective method that greatly improves the sample utility of state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-11-07T10:02:55Z)
• Differentially Private Language Models for Secure Data Sharing [19.918137395199224]
  In this paper, we show how to train a generative language model in a differentially private manner and consequently sampling data from it. Using natural language prompts and a new prompt-mismatch loss, we are able to create highly accurate and fluent textual datasets. We perform thorough experiments indicating that our synthetic datasets do not leak information from our original data and are of high language quality.
  arXiv  Detail & Related papers  (2022-10-25T11:12:56Z)
• Just Fine-tune Twice: Selective Differential Privacy for Large Language Models [69.66654761324702]
  We propose a simple yet effective just-fine-tune-twice privacy mechanism to achieve SDP for large Transformer-based language models. Experiments show that our models achieve strong performance while staying robust to the canary insertion attack.
  arXiv  Detail & Related papers  (2022-04-15T22:36:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.