Tight Bounds for Machine Unlearning via Differential Privacy
- URL: http://arxiv.org/abs/2309.00886v1
- Date: Sat, 2 Sep 2023 09:55:29 GMT
- Title: Tight Bounds for Machine Unlearning via Differential Privacy
- Authors: Yiyang Huang, Cl\'ement L. Canonne
- Abstract summary: We consider the so-called "right to be forgotten" by requiring that a trained model should be able to "unlearn" a number of points from the training data.
We obtain tight bounds on the deletion capacity achievable by DP-based machine unlearning algorithms.
- Score: 0.7252027234425334
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We consider the formulation of "machine unlearning" of Sekhari, Acharya,
Kamath, and Suresh (NeurIPS 2021), which formalizes the so-called "right to be
forgotten" by requiring that a trained model, upon request, should be able to
"unlearn" a number of points from the training data, as if they had never been
included in the first place. Sekhari et al. established some positive and
negative results about the number of data points that can be successfully
unlearnt by a trained model without impacting the model's accuracy (the
"deletion capacity"), showing that machine unlearning could be achieved by
using differentially private (DP) algorithms. However, their results left open
a gap between upper and lower bounds on the deletion capacity of these
algorithms: our work fully closes this gap, obtaining tight bounds on the
deletion capacity achievable by DP-based machine unlearning algorithms.
Related papers
- MUSE: Machine Unlearning Six-Way Evaluation for Language Models [109.76505405962783]
Language models (LMs) are trained on vast amounts of text data, which may include private and copyrighted content.
We propose MUSE, a comprehensive machine unlearning evaluation benchmark.
We benchmark how effectively eight popular unlearning algorithms can unlearn Harry Potter books and news articles.
arXiv Detail & Related papers (2024-07-08T23:47:29Z) - Data Shapley in One Training Run [88.59484417202454]
Data Shapley provides a principled framework for attributing data's contribution within machine learning contexts.
Existing approaches require re-training models on different data subsets, which is computationally intensive.
This paper introduces In-Run Data Shapley, which addresses these limitations by offering scalable data attribution for a target model of interest.
arXiv Detail & Related papers (2024-06-16T17:09:24Z) - Loss-Free Machine Unlearning [51.34904967046097]
We present a machine unlearning approach that is both retraining- and label-free.
Retraining-free approaches often utilise Fisher information, which is derived from the loss and requires labelled data which may not be available.
We present an extension to the Selective Synaptic Dampening algorithm, substituting the diagonal of the Fisher information matrix for the gradient of the l2 norm of the model output to approximate sensitivity.
arXiv Detail & Related papers (2024-02-29T16:15:34Z) - Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning
Interference with Gradient Projection [56.292071534857946]
Recent data-privacy laws have sparked interest in machine unlearning.
Challenge is to discard information about the forget'' data without altering knowledge about remaining dataset.
We adopt a projected-gradient based learning method, named as Projected-Gradient Unlearning (PGU)
We provide empirically evidence to demonstrate that our unlearning method can produce models that behave similar to models retrained from scratch across various metrics even when the training dataset is no longer accessible.
arXiv Detail & Related papers (2023-12-07T07:17:24Z) - Random Relabeling for Efficient Machine Unlearning [8.871042314510788]
Individuals' right to retract personal data and relevant data privacy regulations pose great challenges to machine learning.
We propose unlearning scheme random relabeling to efficiently deal with sequential data removal requests.
A less constraining removal certification method based on probability distribution similarity with naive unlearning is also proposed.
arXiv Detail & Related papers (2023-05-21T02:37:26Z) - Boundary Unlearning [5.132489421775161]
We propose Boundary Unlearning, a rapid yet effective way to unlearn an entire class from a trained machine learning model.
We extensively evaluate Boundary Unlearning on image classification and face recognition tasks, with an expected speed-up of $17times$ and $19times$, respectively.
arXiv Detail & Related papers (2023-03-21T03:33:18Z) - Forget Unlearning: Towards True Data-Deletion in Machine Learning [18.656957502454592]
We show that unlearning is not equivalent to data deletion and does not guarantee the "right to be forgotten"
We propose an accurate, computationally efficient, and secure data-deletion machine learning algorithm in the online setting.
arXiv Detail & Related papers (2022-10-17T10:06:11Z) - Evaluating Machine Unlearning via Epistemic Uncertainty [78.27542864367821]
This work presents an evaluation of Machine Unlearning algorithms based on uncertainty.
This is the first definition of a general evaluation of our best knowledge.
arXiv Detail & Related papers (2022-08-23T09:37:31Z) - On the Necessity of Auditable Algorithmic Definitions for Machine
Unlearning [13.149070833843133]
Machine unlearning, i.e. having a model forget about some of its training data, has become increasingly important as privacy legislation promotes variants of the right-to-be-forgotten.
We first show that the definition that underlies approximate unlearning, which seeks to prove the approximately unlearned model is close to an exactly retrained model, is incorrect because one can obtain the same model using different datasets.
We then turn to exact unlearning approaches and ask how to verify their claims of unlearning.
arXiv Detail & Related papers (2021-10-22T16:16:56Z) - Machine Unlearning of Features and Labels [72.81914952849334]
We propose first scenarios for unlearning and labels in machine learning models.
Our approach builds on the concept of influence functions and realizes unlearning through closed-form updates of model parameters.
arXiv Detail & Related papers (2021-08-26T04:42:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.