The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining

• URL: http://arxiv.org/abs/2203.07320v1
• Date: Mon, 14 Mar 2022 17:22:40 GMT
• Title: The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining
• Authors: Yi Liu, Lei Xu, Xingliang Yuan, Cong Wang, Bo Li
• Abstract summary: We propose a rapid retraining approach to fully erase data samples from a trained FL model. Our formal convergence and complexity analysis demonstrate that our design can preserve model utility with high efficiency.
• Score: 22.16510303054159
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In Machine Learning, the emergence of \textit{the right to be forgotten} gave birth to a paradigm named \textit{machine unlearning}, which enables data holders to proactively erase their data from a trained model. Existing machine unlearning techniques focus on centralized training, where access to all holders' training data is a must for the server to conduct the unlearning process. It remains largely underexplored about how to achieve unlearning when full access to all training data becomes unavailable. One noteworthy example is Federated Learning (FL), where each participating data holder trains locally, without sharing their training data to the central server. In this paper, we investigate the problem of machine unlearning in FL systems. We start with a formal definition of the unlearning problem in FL and propose a rapid retraining approach to fully erase data samples from a trained FL model. The resulting design allows data holders to jointly conduct the unlearning process efficiently while keeping their training data locally. Our formal convergence and complexity analysis demonstrate that our design can preserve model utility with high efficiency. Extensive evaluations on four real-world datasets illustrate the effectiveness and performance of our proposed realization.

Related papers

• ConDa: Fast Federated Unlearning with Contribution Dampening [46.074452659791575]
  ConDa is a framework that performs efficient unlearning by tracking down the parameters which affect the global model for each client. We perform experiments on multiple datasets and demonstrate that ConDa is effective to forget a client's data.
  arXiv  Detail & Related papers  (2024-10-05T12:45:35Z)
• FedQUIT: On-Device Federated Unlearning via a Quasi-Competent Virtual Teacher [4.291269657919828]
  Federated Learning (FL) systems enable the collaborative training of machine learning models without requiring centralized collection of individual data. We propose FedQUIT, a novel algorithm that uses knowledge distillation to scrub the contribution of the data to forget from an FL global model.
  arXiv  Detail & Related papers  (2024-08-14T14:36:28Z)
• Update Selective Parameters: Federated Machine Unlearning Based on Model Explanation [46.86767774669831]
  We propose a more effective and efficient federated unlearning scheme based on the concept of model explanation. We select the most influential channels within an already-trained model for the data that need to be unlearned.
  arXiv  Detail & Related papers  (2024-06-18T11:43:20Z)
• Blockchain-enabled Trustworthy Federated Unlearning [50.01101423318312]
  Federated unlearning is a promising paradigm for protecting the data ownership of distributed clients. Existing works require central servers to retain the historical model parameters from distributed clients. This paper proposes a new blockchain-enabled trustworthy federated unlearning framework.
  arXiv  Detail & Related papers  (2024-01-29T07:04:48Z)
• Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening [51.34904967046097]
  Selective Synaptic Dampening (SSD) is a fast, performant, and does not require long-term storage of the training data. We present a novel two-step, post hoc, retrain-free approach to machine unlearning which is fast, performant, and does not require long-term storage of the training data.
  arXiv  Detail & Related papers  (2023-08-15T11:30:45Z)
• Fair Machine Unlearning: Data Removal while Mitigating Disparities [5.724350004671127]
  Right to be forgotten is core principle outlined by EU's General Regulation. "Forgetting" can be naively achieved by retraining on remaining data. "Unlearning" impacts other properties critical to real-world applications such as fairness.
  arXiv  Detail & Related papers  (2023-07-27T10:26:46Z)
• Federated Unlearning: How to Efficiently Erase a Client in FL? [9.346673106489742]
  We propose a method to erase a client by removing the influence of their entire local data from the trained global model. Our unlearning method achieves comparable performance as the gold standard unlearning method of federated retraining from scratch. Unlike prior works, our unlearning method neither requires global access to the data used for training nor the history of the parameter updates to be stored by the server or any of the clients.
  arXiv  Detail & Related papers  (2022-07-12T13:24:23Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Federated Unlearning with Knowledge Distillation [9.666514931140707]
  Federated Learning (FL) is designed to protect the data privacy of each client during the training process. With the recent legislation on right to be forgotten, it is crucially essential for the FL model to possess the ability to forget what it has learned from each client. We propose a novel federated unlearning method to eliminate a client's contribution by subtracting the accumulated historical updates from the model.
  arXiv  Detail & Related papers  (2022-01-24T03:56:20Z)
• Decentralized Federated Learning Preserves Model and Data Privacy [77.454688257702]
  We propose a fully decentralized approach, which allows to share knowledge between trained models. Students are trained on the output of their teachers via synthetically generated input data. The results show that an untrained student model, trained on the teachers output reaches comparable F1-scores as the teacher.
  arXiv  Detail & Related papers  (2021-02-01T14:38:54Z)
• A Principled Approach to Data Valuation for Federated Learning [73.19984041333599]
  Federated learning (FL) is a popular technique to train machine learning (ML) models on decentralized data sources. The Shapley value (SV) defines a unique payoff scheme that satisfies many desiderata for a data value notion. This paper proposes a variant of the SV amenable to FL, which we call the federated Shapley value.
  arXiv  Detail & Related papers  (2020-09-14T04:37:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.