Machine Unlearning of Features and Labels
- URL: http://arxiv.org/abs/2108.11577v4
- Date: Mon, 7 Aug 2023 12:33:20 GMT
- Title: Machine Unlearning of Features and Labels
- Authors: Alexander Warnecke, Lukas Pirch, Christian Wressnegger and Konrad
Rieck
- Abstract summary: We propose first scenarios for unlearning and labels in machine learning models.
Our approach builds on the concept of influence functions and realizes unlearning through closed-form updates of model parameters.
- Score: 72.81914952849334
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Removing information from a machine learning model is a non-trivial task that
requires to partially revert the training process. This task is unavoidable
when sensitive data, such as credit card numbers or passwords, accidentally
enter the model and need to be removed afterwards. Recently, different concepts
for machine unlearning have been proposed to address this problem. While these
approaches are effective in removing individual data points, they do not scale
to scenarios where larger groups of features and labels need to be reverted. In
this paper, we propose the first method for unlearning features and labels. Our
approach builds on the concept of influence functions and realizes unlearning
through closed-form updates of model parameters. It enables to adapt the
influence of training data on a learning model retrospectively, thereby
correcting data leaks and privacy issues. For learning models with strongly
convex loss functions, our method provides certified unlearning with
theoretical guarantees. For models with non-convex losses, we empirically show
that unlearning features and labels is effective and significantly faster than
other strategies.
Related papers
- Update Selective Parameters: Federated Machine Unlearning Based on Model Explanation [46.86767774669831]
We propose a more effective and efficient federated unlearning scheme based on the concept of model explanation.
We select the most influential channels within an already-trained model for the data that need to be unlearned.
arXiv Detail & Related papers (2024-06-18T11:43:20Z) - Towards Efficient Target-Level Machine Unlearning Based on Essential Graph [18.35868679190816]
Existing studies of machine unlearning mainly focus on unlearning requests that forget a cluster of instances or all instances from one class.
We propose a more effective and efficient unlearning scheme that focuses on removing partial targets from the model.
Experiments with different training models on various datasets demonstrate the effectiveness of the proposed approach.
arXiv Detail & Related papers (2024-06-16T14:17:13Z) - The Frontier of Data Erasure: Machine Unlearning for Large Language Models [56.26002631481726]
Large Language Models (LLMs) are foundational to AI advancements.
LLMs pose risks by potentially memorizing and disseminating sensitive, biased, or copyrighted information.
Machine unlearning emerges as a cutting-edge solution to mitigate these concerns.
arXiv Detail & Related papers (2024-03-23T09:26:15Z) - Efficient Knowledge Deletion from Trained Models through Layer-wise
Partial Machine Unlearning [2.3496568239538083]
This paper introduces a novel class of machine unlearning algorithms.
First method is partial amnesiac unlearning, integration of layer-wise pruning with amnesiac unlearning.
Second method assimilates layer-wise partial-updates into label-flipping and optimization-based unlearning.
arXiv Detail & Related papers (2024-03-12T12:49:47Z) - An Information Theoretic Approach to Machine Unlearning [45.600917449314444]
Key challenge in unlearning is forgetting the necessary data in a timely manner, while preserving model performance.
In this work, we address the zero-shot unlearning scenario, whereby an unlearning algorithm must be able to remove data given only a trained model and the data to be forgotten.
We derive a simple but principled zero-shot unlearning method based on the geometry of the model.
arXiv Detail & Related papers (2024-02-02T13:33:30Z) - Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning
Interference with Gradient Projection [56.292071534857946]
Recent data-privacy laws have sparked interest in machine unlearning.
Challenge is to discard information about the forget'' data without altering knowledge about remaining dataset.
We adopt a projected-gradient based learning method, named as Projected-Gradient Unlearning (PGU)
We provide empirically evidence to demonstrate that our unlearning method can produce models that behave similar to models retrained from scratch across various metrics even when the training dataset is no longer accessible.
arXiv Detail & Related papers (2023-12-07T07:17:24Z) - Unlearn What You Want to Forget: Efficient Unlearning for LLMs [92.51670143929056]
Large language models (LLMs) have achieved significant progress from pre-training on and memorizing a wide range of textual data.
This process might suffer from privacy issues and violations of data protection regulations.
We propose an efficient unlearning framework that could efficiently update LLMs without having to retrain the whole model after data removals.
arXiv Detail & Related papers (2023-10-31T03:35:59Z) - AI Model Disgorgement: Methods and Choices [127.54319351058167]
We introduce a taxonomy of possible disgorgement methods that are applicable to modern machine learning systems.
We investigate the meaning of "removing the effects" of data in the trained model in a way that does not require retraining from scratch.
arXiv Detail & Related papers (2023-04-07T08:50:18Z) - Learning to Unlearn: Instance-wise Unlearning for Pre-trained
Classifiers [71.70205894168039]
We consider instance-wise unlearning, of which the goal is to delete information on a set of instances from a pre-trained model.
We propose two methods that reduce forgetting on the remaining data: 1) utilizing adversarial examples to overcome forgetting at the representation-level and 2) leveraging weight importance metrics to pinpoint network parameters guilty of propagating unwanted information.
arXiv Detail & Related papers (2023-01-27T07:53:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.