Training Natural Language Processing Models on Encrypted Text for Enhanced Privacy

• URL: http://arxiv.org/abs/2305.03497v1
• Date: Wed, 3 May 2023 00:37:06 GMT
• Title: Training Natural Language Processing Models on Encrypted Text for Enhanced Privacy
• Authors: Davut Emre Tasar, Ceren Ocal Tasar
• Abstract summary: We propose a method for training NLP models on encrypted text data to mitigate data privacy concerns. Our results indicate that both encrypted and non-encrypted models achieve comparable performance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the increasing use of cloud-based services for training and deploying machine learning models, data privacy has become a major concern. This is particularly important for natural language processing (NLP) models, which often process sensitive information such as personal communications and confidential documents. In this study, we propose a method for training NLP models on encrypted text data to mitigate data privacy concerns while maintaining similar performance to models trained on non-encrypted data. We demonstrate our method using two different architectures, namely Doc2Vec+XGBoost and Doc2Vec+LSTM, and evaluate the models on the 20 Newsgroups dataset. Our results indicate that both encrypted and non-encrypted models achieve comparable performance, suggesting that our encryption method is effective in preserving data privacy without sacrificing model accuracy. In order to replicate our experiments, we have provided a Colab notebook at the following address: https://t.ly/lR-TP

Related papers

• Robust Utility-Preserving Text Anonymization Based on Large Language Models [80.5266278002083]
  Text anonymization is crucial for sharing sensitive data while maintaining privacy. Existing techniques face the emerging challenges of re-identification attack ability of Large Language Models. This paper proposes a framework composed of three LLM-based components -- a privacy evaluator, a utility evaluator, and an optimization component.
  arXiv  Detail & Related papers  (2024-07-16T14:28:56Z)
• FewFedPIT: Towards Privacy-preserving and Few-shot Federated Instruction Tuning [54.26614091429253]
  Federated instruction tuning (FedIT) is a promising solution, by consolidating collaborative training across multiple data owners. FedIT encounters limitations such as scarcity of instructional data and risk of exposure to training data extraction attacks. We propose FewFedPIT, designed to simultaneously enhance privacy protection and model performance of federated few-shot learning.
  arXiv  Detail & Related papers  (2024-03-10T08:41:22Z)
• SentinelLMs: Encrypted Input Adaptation and Fine-tuning of Language Models for Private and Secure Inference [6.0189674528771]
  This paper addresses the privacy and security concerns associated with deep neural language models. Deep neural language models serve as crucial components in various modern AI-based applications. We propose a novel method to adapt and fine-tune transformer-based language models on passkey-encrypted user-specific text.
  arXiv  Detail & Related papers  (2023-12-28T19:55:11Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• Recovering from Privacy-Preserving Masking with Large Language Models [14.828717714653779]
  We use large language models (LLMs) to suggest substitutes of masked tokens. We show that models trained on the obfuscation corpora are able to achieve comparable performance with the ones trained on the original data.
  arXiv  Detail & Related papers  (2023-09-12T16:39:41Z)
• Robust Representation Learning for Privacy-Preserving Machine Learning: A Multi-Objective Autoencoder Approach [0.9831489366502302]
  We propose a robust representation learning framework for privacy-preserving machine learning (ppML) Our method centers on training autoencoders in a multi-objective manner and then concatenating the latent and learned features from the encoding part as the encoded form of our data. With our proposed framework, we can share our data and use third party tools without being under the threat of revealing its original form.
  arXiv  Detail & Related papers  (2023-09-08T16:41:25Z)
• Just Fine-tune Twice: Selective Differential Privacy for Large Language Models [69.66654761324702]
  We propose a simple yet effective just-fine-tune-twice privacy mechanism to achieve SDP for large Transformer-based language models. Experiments show that our models achieve strong performance while staying robust to the canary insertion attack.
  arXiv  Detail & Related papers  (2022-04-15T22:36:55Z)
• Extracting Training Data from Large Language Models [78.3839333127544]
  This paper demonstrates that an adversary can perform a training data extraction attack to recover individual training examples by querying the language model. We demonstrate our attack on GPT-2, a language model trained on scrapes of the public Internet, and are able to extract hundreds of verbatim text sequences from the model's training data.
  arXiv  Detail & Related papers  (2020-12-14T18:39:09Z)
• Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
  We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting. Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking. We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
  arXiv  Detail & Related papers  (2020-10-24T11:55:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.