Machine Unlearning of Features and Labels
- URL: http://arxiv.org/abs/2108.11577v4
- Date: Mon, 7 Aug 2023 12:33:20 GMT
- Title: Machine Unlearning of Features and Labels
- Authors: Alexander Warnecke, Lukas Pirch, Christian Wressnegger and Konrad
Rieck
- Abstract summary: We propose first scenarios for unlearning and labels in machine learning models.
Our approach builds on the concept of influence functions and realizes unlearning through closed-form updates of model parameters.
- Score: 72.81914952849334
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Removing information from a machine learning model is a non-trivial task that
requires to partially revert the training process. This task is unavoidable
when sensitive data, such as credit card numbers or passwords, accidentally
enter the model and need to be removed afterwards. Recently, different concepts
for machine unlearning have been proposed to address this problem. While these
approaches are effective in removing individual data points, they do not scale
to scenarios where larger groups of features and labels need to be reverted. In
this paper, we propose the first method for unlearning features and labels. Our
approach builds on the concept of influence functions and realizes unlearning
through closed-form updates of model parameters. It enables to adapt the
influence of training data on a learning model retrospectively, thereby
correcting data leaks and privacy issues. For learning models with strongly
convex loss functions, our method provides certified unlearning with
theoretical guarantees. For models with non-convex losses, we empirically show
that unlearning features and labels is effective and significantly faster than
other strategies.
Related papers
- Machine Unlearning on Pre-trained Models by Residual Feature Alignment Using LoRA [15.542668474378633]
We propose a novel and efficient machine unlearning method on pre-trained models.
We leverage LoRA to decompose the model's intermediate features into pre-trained features and residual features.
The method aims to learn the zero residuals on the retained set and shifted residuals on the unlearning set.
arXiv Detail & Related papers (2024-11-13T08:56:35Z) - RESTOR: Knowledge Recovery through Machine Unlearning [71.75834077528305]
Large language models trained on web-scale corpora can memorize undesirable datapoints.
Many machine unlearning methods have been proposed that aim to 'erase' these datapoints from trained models.
We propose the RESTOR framework for machine unlearning based on the following dimensions.
arXiv Detail & Related papers (2024-10-31T20:54:35Z) - Towards Efficient Target-Level Machine Unlearning Based on Essential Graph [18.35868679190816]
Existing studies of machine unlearning mainly focus on unlearning requests that forget a cluster of instances or all instances from one class.
We propose a more effective and efficient unlearning scheme that focuses on removing partial targets from the model.
Experiments with different training models on various datasets demonstrate the effectiveness of the proposed approach.
arXiv Detail & Related papers (2024-06-16T14:17:13Z) - Efficient Knowledge Deletion from Trained Models through Layer-wise
Partial Machine Unlearning [2.3496568239538083]
This paper introduces a novel class of machine unlearning algorithms.
First method is partial amnesiac unlearning, integration of layer-wise pruning with amnesiac unlearning.
Second method assimilates layer-wise partial-updates into label-flipping and optimization-based unlearning.
arXiv Detail & Related papers (2024-03-12T12:49:47Z) - An Information Theoretic Approach to Machine Unlearning [45.600917449314444]
Key challenge in unlearning is forgetting the necessary data in a timely manner, while preserving model performance.
In this work, we address the zero-shot unlearning scenario, whereby an unlearning algorithm must be able to remove data given only a trained model and the data to be forgotten.
We derive a simple but principled zero-shot unlearning method based on the geometry of the model.
arXiv Detail & Related papers (2024-02-02T13:33:30Z) - Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning
Interference with Gradient Projection [56.292071534857946]
Recent data-privacy laws have sparked interest in machine unlearning.
Challenge is to discard information about the forget'' data without altering knowledge about remaining dataset.
We adopt a projected-gradient based learning method, named as Projected-Gradient Unlearning (PGU)
We provide empirically evidence to demonstrate that our unlearning method can produce models that behave similar to models retrained from scratch across various metrics even when the training dataset is no longer accessible.
arXiv Detail & Related papers (2023-12-07T07:17:24Z) - Unlearn What You Want to Forget: Efficient Unlearning for LLMs [92.51670143929056]
Large language models (LLMs) have achieved significant progress from pre-training on and memorizing a wide range of textual data.
This process might suffer from privacy issues and violations of data protection regulations.
We propose an efficient unlearning framework that could efficiently update LLMs without having to retrain the whole model after data removals.
arXiv Detail & Related papers (2023-10-31T03:35:59Z) - AI Model Disgorgement: Methods and Choices [127.54319351058167]
We introduce a taxonomy of possible disgorgement methods that are applicable to modern machine learning systems.
We investigate the meaning of "removing the effects" of data in the trained model in a way that does not require retraining from scratch.
arXiv Detail & Related papers (2023-04-07T08:50:18Z) - Learning to Unlearn: Instance-wise Unlearning for Pre-trained
Classifiers [71.70205894168039]
We consider instance-wise unlearning, of which the goal is to delete information on a set of instances from a pre-trained model.
We propose two methods that reduce forgetting on the remaining data: 1) utilizing adversarial examples to overcome forgetting at the representation-level and 2) leveraging weight importance metrics to pinpoint network parameters guilty of propagating unwanted information.
arXiv Detail & Related papers (2023-01-27T07:53:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.