Towards Probabilistic Verification of Machine Unlearning
- URL: http://arxiv.org/abs/2003.04247v2
- Date: Tue, 1 Dec 2020 16:01:10 GMT
- Title: Towards Probabilistic Verification of Machine Unlearning
- Authors: David Marco Sommer, Liwei Song, Sameer Wagh, Prateek Mittal
- Abstract summary: We propose a formal framework to study the design of verification mechanisms for data deletion requests.
We show that our approach has minimal effect on the machine learning service's accuracy but provides high confidence verification of unlearning.
- Score: 30.892906429582904
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The right to be forgotten, also known as the right to erasure, is the right
of individuals to have their data erased from an entity storing it. The status
of this long held notion was legally solidified recently by the General Data
Protection Regulation (GDPR) in the European Union. Consequently, there is a
need for mechanisms whereby users can verify if service providers comply with
their deletion requests. In this work, we take the first step in proposing a
formal framework to study the design of such verification mechanisms for data
deletion requests -- also known as machine unlearning -- in the context of
systems that provide machine learning as a service (MLaaS). Our framework
allows the rigorous quantification of any verification mechanism based on
standard hypothesis testing. Furthermore, we propose a novel backdoor-based
verification mechanism and demonstrate its effectiveness in certifying data
deletion with high confidence, thus providing a basis for quantitatively
inferring machine unlearning.
We evaluate our approach over a range of network architectures such as
multi-layer perceptrons (MLP), convolutional neural networks (CNN), residual
networks (ResNet), and long short-term memory (LSTM), as well as over 5
different datasets. We demonstrate that our approach has minimal effect on the
ML service's accuracy but provides high confidence verification of unlearning.
Our proposed mechanism works even if only a handful of users employ our system
to ascertain compliance with data deletion requests. In particular, with just
5% of users participating, modifying half their data with a backdoor, and with
merely 30 test queries, our verification mechanism has both false positive and
false negative ratios below $10^{-3}$. We also show the effectiveness of our
approach by testing it against an adaptive adversary that uses a
state-of-the-art backdoor defense method.
Related papers
- Pseudo-Probability Unlearning: Towards Efficient and Privacy-Preserving Machine Unlearning [59.29849532966454]
We propose PseudoProbability Unlearning (PPU), a novel method that enables models to forget data to adhere to privacy-preserving manner.
Our method achieves over 20% improvements in forgetting error compared to the state-of-the-art.
arXiv Detail & Related papers (2024-11-04T21:27:06Z) - Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy [65.80757820884476]
We expose a critical yet underexplored vulnerability in the deployment of unlearning systems.
We present a threat model where an attacker can degrade model accuracy by submitting adversarial unlearning requests for data not present in the training set.
We evaluate various verification mechanisms to detect the legitimacy of unlearning requests and reveal the challenges in verification.
arXiv Detail & Related papers (2024-10-12T16:47:04Z) - Silver Linings in the Shadows: Harnessing Membership Inference for Machine Unlearning [7.557226714828334]
We present a novel unlearning mechanism designed to remove the impact of specific data samples from a neural network.
In achieving this goal, we crafted a novel loss function tailored to eliminate privacy-sensitive information from weights and activation values of the target model.
Our results showcase the superior performance of our approach in terms of unlearning efficacy and latency as well as the fidelity of the primary task.
arXiv Detail & Related papers (2024-07-01T00:20:26Z) - Enhancing Security in Federated Learning through Adaptive
Consensus-Based Model Update Validation [2.28438857884398]
This paper introduces an advanced approach for fortifying Federated Learning (FL) systems against label-flipping attacks.
We propose a consensus-based verification process integrated with an adaptive thresholding mechanism.
Our results indicate a significant mitigation of label-flipping attacks, bolstering the FL system's resilience.
arXiv Detail & Related papers (2024-03-05T20:54:56Z) - Blockchain-enabled Trustworthy Federated Unlearning [50.01101423318312]
Federated unlearning is a promising paradigm for protecting the data ownership of distributed clients.
Existing works require central servers to retain the historical model parameters from distributed clients.
This paper proposes a new blockchain-enabled trustworthy federated unlearning framework.
arXiv Detail & Related papers (2024-01-29T07:04:48Z) - DefectHunter: A Novel LLM-Driven Boosted-Conformer-based Code Vulnerability Detection Mechanism [3.9377491512285157]
DefectHunter is an innovative model for vulnerability identification that employs the Conformer mechanism.
This mechanism fuses self-attention with convolutional networks to capture both local, position-wise features and global, content-based interactions.
arXiv Detail & Related papers (2023-09-27T00:10:29Z) - A Generative Framework for Low-Cost Result Validation of Machine Learning-as-a-Service Inference [4.478182379059458]
Fides is a novel framework for real-time integrity validation of ML-as-a-Service (ML) inference.
Fides features a client-side attack detection model that uses statistical analysis and divergence measurements to identify, with a high likelihood, if the service model is under attack.
We devised a generative adversarial network framework for training the attack detection and re-classification models.
arXiv Detail & Related papers (2023-03-31T19:17:30Z) - Free Lunch for Generating Effective Outlier Supervision [46.37464572099351]
We propose an ultra-effective method to generate near-realistic outlier supervision.
Our proposed textttBayesAug significantly reduces the false positive rate over 12.50% compared with the previous schemes.
arXiv Detail & Related papers (2023-01-17T01:46:45Z) - RelaxLoss: Defending Membership Inference Attacks without Losing Utility [68.48117818874155]
We propose a novel training framework based on a relaxed loss with a more achievable learning target.
RelaxLoss is applicable to any classification model with added benefits of easy implementation and negligible overhead.
Our approach consistently outperforms state-of-the-art defense mechanisms in terms of resilience against MIAs.
arXiv Detail & Related papers (2022-07-12T19:34:47Z) - Federated Learning with Unreliable Clients: Performance Analysis and
Mechanism Design [76.29738151117583]
Federated Learning (FL) has become a promising tool for training effective machine learning models among distributed clients.
However, low quality models could be uploaded to the aggregator server by unreliable clients, leading to a degradation or even a collapse of training.
We model these unreliable behaviors of clients and propose a defensive mechanism to mitigate such a security risk.
arXiv Detail & Related papers (2021-05-10T08:02:27Z) - Identity-Aware Attribute Recognition via Real-Time Distributed Inference
in Mobile Edge Clouds [53.07042574352251]
We design novel models for pedestrian attribute recognition with re-ID in an MEC-enabled camera monitoring system.
We propose a novel inference framework with a set of distributed modules, by jointly considering the attribute recognition and person re-ID.
We then devise a learning-based algorithm for the distributions of the modules of the proposed distributed inference framework.
arXiv Detail & Related papers (2020-08-12T12:03:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.