Understanding Unintended Memorization in Federated Learning

• URL: http://arxiv.org/abs/2006.07490v1
• Date: Fri, 12 Jun 2020 22:10:16 GMT
• Title: Understanding Unintended Memorization in Federated Learning
• Authors: Om Thakkar, Swaroop Ramaswamy, Rajiv Mathews, Fran\c{c}oise Beaufays
• Abstract summary: We show that different components of Federated Learning play an important role in reducing unintended memorization. We also show that training with a strong user-level differential privacy guarantee results in models that exhibit the least amount of unintended memorization.
• Score: 5.32880378510767
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent works have shown that generative sequence models (e.g., language models) have a tendency to memorize rare or unique sequences in the training data. Since useful models are often trained on sensitive data, to ensure the privacy of the training data it is critical to identify and mitigate such unintended memorization. Federated Learning (FL) has emerged as a novel framework for large-scale distributed learning tasks. However, it differs in many aspects from the well-studied central learning setting where all the data is stored at the central server. In this paper, we initiate a formal study to understand the effect of different components of canonical FL on unintended memorization in trained models, comparing with the central learning setting. Our results show that several differing components of FL play an important role in reducing unintended memorization. Specifically, we observe that the clustering of data according to users---which happens by design in FL---has a significant effect in reducing such memorization, and using the method of Federated Averaging for training causes a further reduction. We also show that training with a strong user-level differential privacy guarantee results in models that exhibit the least amount of unintended memorization.

Related papers

• Understanding Memorisation in LLMs: Dynamics, Influencing Factors, and Implications [14.818820873377303]
  We study whether and to what extent large language models (LLMs) have memorised training data. We create an experimental framework that is based on repeatedly exposing LLMs to random strings. We identify factors that make some strings easier to memorise than others, and we identify the role of local prefixes and global context in memorisation.
  arXiv  Detail & Related papers  (2024-07-27T14:00:21Z)
• Causal Estimation of Memorisation Profiles [58.20086589761273]
  Understanding memorisation in language models has practical and societal implications. Memorisation is the causal effect of training with an instance on the model's ability to predict that instance. This paper proposes a new, principled, and efficient method to estimate memorisation based on the difference-in-differences design from econometrics.
  arXiv  Detail & Related papers  (2024-06-06T17:59:09Z)
• MultiConfederated Learning: Inclusive Non-IID Data handling with Decentralized Federated Learning [1.2726316791083532]
  Federated Learning (FL) has emerged as a prominent privacy-preserving technique for enabling use cases like confidential clinical machine learning. FL operates by aggregating models trained by remote devices which owns the data. We propose MultiConfederated Learning: a decentralized FL framework which is designed to handle non-IID data.
  arXiv  Detail & Related papers  (2024-04-20T16:38:26Z)
• Exploring Memorization in Fine-tuned Language Models [53.52403444655213]
  We conduct the first comprehensive analysis to explore language models' memorization during fine-tuning across tasks. Our studies with open-sourced and our own fine-tuned LMs across various tasks indicate that memorization presents a strong disparity among different fine-tuning tasks. We provide an intuitive explanation of this task disparity via sparse coding theory and unveil a strong correlation between memorization and attention score distribution.
  arXiv  Detail & Related papers  (2023-10-10T15:41:26Z)
• Don't Memorize; Mimic The Past: Federated Class Incremental Learning Without Episodic Memory [36.4406505365313]
  This paper presents a framework for federated class incremental learning that utilizes a generative model to synthesize samples from past distributions instead of storing part of past data. The generative model is trained on the server using data-free methods at the end of each task without requesting data from clients.
  arXiv  Detail & Related papers  (2023-07-02T07:06:45Z)
• When Do Curricula Work in Federated Learning? [56.88941905240137]
  We find that curriculum learning largely alleviates non-IIDness. The more disparate the data distributions across clients the more they benefit from learning. We propose a novel client selection technique that benefits from the real-world disparity in the clients.
  arXiv  Detail & Related papers  (2022-12-24T11:02:35Z)
• Towards Differential Relational Privacy and its use in Question Answering [109.4452196071872]
  Memorization of relation between entities in a dataset can lead to privacy issues when using a trained question answering model. We quantify this phenomenon and provide a possible definition of Differential Privacy (DPRP) We illustrate concepts in experiments with largescale models for Question Answering.
  arXiv  Detail & Related papers  (2022-03-30T22:59:24Z)
• Comparative assessment of federated and centralized machine learning [0.0]
  Federated Learning (FL) is a privacy preserving machine learning scheme, where training happens with data federated across devices. In this paper, we discuss the various factors that affect the federated learning training, because of the non-IID distributed nature of the data. We show that federated learning does have an advantage in cost when the model sizes to be trained are not reasonably large.
  arXiv  Detail & Related papers  (2022-02-03T11:20:47Z)
• Counterfactual Memorization in Neural Language Models [91.8747020391287]
  Modern neural language models that are widely used in various NLP tasks risk memorizing sensitive information from their training data. An open question in previous studies of language model memorization is how to filter out "common" memorization. We formulate a notion of counterfactual memorization which characterizes how a model's predictions change if a particular document is omitted during training.
  arXiv  Detail & Related papers  (2021-12-24T04:20:57Z)
• Reasoning-Modulated Representations [85.08205744191078]
  We study a common setting where our task is not purely opaque. Our approach paves the way for a new class of data-efficient representation learning.
  arXiv  Detail & Related papers  (2021-07-19T13:57:13Z)
• Memory-Associated Differential Learning [10.332918082271153]
  We propose a novel learning paradigm called Memory-Associated Differential (MAD) Learning. We first introduce an additional component called Memory to memorize all the training data. Then we learn the differences of labels as well as the associations of features in the combination of a differential equation and some sampling methods. In the evaluating phase, we predict unknown labels by inferencing from the memorized facts plus the learnt differences and associations in a geometrically meaningful manner.
  arXiv  Detail & Related papers  (2021-02-10T03:48:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.