Recovering Private Text in Federated Learning of Language Models

• URL: http://arxiv.org/abs/2205.08514v1
• Date: Tue, 17 May 2022 17:38:37 GMT
• Title: Recovering Private Text in Federated Learning of Language Models
• Authors: Samyak Gupta, Yangsibo Huang, Zexuan Zhong, Tianyu Gao, Kai Li, Danqi Chen
• Abstract summary: Federated learning allows distributed users to collaboratively train a model while keeping each user's data private. We present a novel attack method FILM for federated learning of language models. We show the feasibility of recovering text from large batch sizes of up to 128 sentences.
• Score: 30.646865969760412
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Federated learning allows distributed users to collaboratively train a model while keeping each user's data private. Recently, a growing body of work has demonstrated that an eavesdropping attacker can effectively recover image data from gradients transmitted during federated learning. However, little progress has been made in recovering text data. In this paper, we present a novel attack method FILM for federated learning of language models -- for the first time, we show the feasibility of recovering text from large batch sizes of up to 128 sentences. Different from image-recovery methods which are optimized to match gradients, we take a distinct approach that first identifies a set of words from gradients and then directly reconstructs sentences based on beam search and a prior-based reordering strategy. The key insight of our attack is to leverage either prior knowledge in pre-trained language models or memorization during training. Despite its simplicity, we demonstrate that FILM can work well with several large-scale datasets -- it can extract single sentences with high fidelity even for large batch sizes and recover multiple sentences from the batch successfully if the attack is applied iteratively. We hope our results can motivate future work in developing stronger attacks as well as new defense methods for training language models in federated learning. Our code is publicly available at https://github.com/Princeton-SysML/FILM.

Related papers

• CLEFT: Language-Image Contrastive Learning with Efficient Large Language Model and Prompt Fine-Tuning [4.004641316826348]
  We introduce a novel language-image Contrastive Learning method with an Efficient large language model and prompt Fine-Tuning (CLEFT) Our method demonstrates state-of-the-art performance on multiple chest X-ray and mammography datasets. The proposed parameter efficient framework can reduce the total trainable model size by 39% and reduce the trainable language model to only 4% compared with the current BERT encoder.
  arXiv  Detail & Related papers  (2024-07-30T17:57:32Z)
• Learning High-Quality and General-Purpose Phrase Representations [9.246374019271938]
  Phrase representations play an important role in data science and natural language processing. Current state-of-the-art method involves fine-tuning pre-trained language models for phrasal embeddings. We propose an improved framework to learn phrase representations in a context-free fashion.
  arXiv  Detail & Related papers  (2024-01-18T22:32:31Z)
• Scalable Extraction of Training Data from (Production) Language Models [93.7746567808049]
  This paper studies extractable memorization: training data that an adversary can efficiently extract by querying a machine learning model without prior knowledge of the training dataset. We show an adversary can extract gigabytes of training data from open-source language models like Pythia or GPT-Neo, semi-open models like LLaMA or Falcon, and closed models like ChatGPT.
  arXiv  Detail & Related papers  (2023-11-28T18:47:03Z)
• Generative Negative Text Replay for Continual Vision-Language Pretraining [95.2784858069843]
  Vision-language pre-training has attracted increasing attention recently. Massive data are usually collected in a streaming fashion. We propose a multi-modal knowledge distillation between images and texts to align the instance-wise prediction between old and new models.
  arXiv  Detail & Related papers  (2022-10-31T13:42:21Z)
• Language Model Pre-Training with Sparse Latent Typing [66.75786739499604]
  We propose a new pre-training objective, Sparse Latent Typing, which enables the model to sparsely extract sentence-level keywords with diverse latent types. Experimental results show that our model is able to learn interpretable latent type categories in a self-supervised manner without using any external knowledge.
  arXiv  Detail & Related papers  (2022-10-23T00:37:08Z)
• LAMP: Extracting Text from Gradients with Language Model Priors [9.242965489146398]
  Recent work shows that sensitive user data can be reconstructed from gradient updates, breaking the key privacy promise of federated learning. We propose LAMP, a novel attack tailored to textual data, that successfully reconstructs original text from gradients.
  arXiv  Detail & Related papers  (2022-02-17T18:49:25Z)
• Paraphrastic Representations at Scale [134.41025103489224]
  We release trained models for English, Arabic, German, French, Spanish, Russian, Turkish, and Chinese languages. We train these models on large amounts of data, achieving significantly improved performance from the original papers.
  arXiv  Detail & Related papers  (2021-04-30T16:55:28Z)
• Extracting Training Data from Large Language Models [78.3839333127544]
  This paper demonstrates that an adversary can perform a training data extraction attack to recover individual training examples by querying the language model. We demonstrate our attack on GPT-2, a language model trained on scrapes of the public Internet, and are able to extract hundreds of verbatim text sequences from the model's training data.
  arXiv  Detail & Related papers  (2020-12-14T18:39:09Z)
• Pre-training via Paraphrasing [96.79972492585112]
  We introduce MARGE, a pre-trained sequence-to-sequence model learned with an unsupervised multi-lingual paraphrasing objective. We show it is possible to jointly learn to do retrieval and reconstruction, given only a random initialization. For example, with no additional task-specific training we achieve BLEU scores of up to 35.8 for document translation.
  arXiv  Detail & Related papers  (2020-06-26T14:43:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.