DP-UTIL: Comprehensive Utility Analysis of Differential Privacy in
Machine Learning
- URL: http://arxiv.org/abs/2112.12998v1
- Date: Fri, 24 Dec 2021 08:40:28 GMT
- Title: DP-UTIL: Comprehensive Utility Analysis of Differential Privacy in
Machine Learning
- Authors: Ismat Jarin and Birhanu Eshete
- Abstract summary: Differential Privacy (DP) has emerged as a rigorous formalism to reason about privacy leakage.
In machine learning (ML), DP has been employed to limit/disclosure of training examples.
For deep neural networks, gradient perturbation results in lowest privacy leakage.
- Score: 3.822543555265593
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Differential Privacy (DP) has emerged as a rigorous formalism to reason about
quantifiable privacy leakage. In machine learning (ML), DP has been employed to
limit inference/disclosure of training examples. Prior work leveraged DP across
the ML pipeline, albeit in isolation, often focusing on mechanisms such as
gradient perturbation. In this paper, we present, DP-UTIL, a holistic utility
analysis framework of DP across the ML pipeline with focus on input
perturbation, objective perturbation, gradient perturbation, output
perturbation, and prediction perturbation. Given an ML task on
privacy-sensitive data, DP-UTIL enables a ML privacy practitioner perform
holistic comparative analysis on the impact of DP in these five perturbation
spots, measured in terms of model utility loss, privacy leakage, and the number
of truly revealed training samples. We evaluate DP-UTIL over classification
tasks on vision, medical, and financial datasets, using two representative
learning algorithms (logistic regression and deep neural network) against
membership inference attack as a case study attack. One of the highlights of
our results is that prediction perturbation consistently achieves the lowest
utility loss on all models across all datasets. In logistic regression models,
objective perturbation results in lowest privacy leakage compared to other
perturbation techniques. For deep neural networks, gradient perturbation
results in lowest privacy leakage. Moreover, our results on true revealed
records suggest that as privacy leakage increases a differentially private
model reveals more number of member samples. Overall, our findings suggest that
to make informed decisions as to which perturbation mechanism to use, a ML
privacy practitioner needs to examine the dynamics between optimization
techniques (convex vs. non-convex), perturbation mechanisms, number of classes,
and privacy budget.
Related papers
- Differentially Private Log-Location-Scale Regression Using Functional Mechanism [1.2277343096128712]
This article introduces differentially private log-location-scale (DP-LLS) regression models, which incorporate differential privacy into LLS regression.
We will derive the sensitivities utilized to determine the magnitude of the injected noise and prove that the proposed DP-LLS models satisfy $epsilon$-differential privacy.
arXiv Detail & Related papers (2024-04-12T04:14:08Z) - Initialization Matters: Privacy-Utility Analysis of Overparameterized
Neural Networks [72.51255282371805]
We prove a privacy bound for the KL divergence between model distributions on worst-case neighboring datasets.
We find that this KL privacy bound is largely determined by the expected squared gradient norm relative to model parameters during training.
arXiv Detail & Related papers (2023-10-31T16:13:22Z) - A Differentially Private Weighted Empirical Risk Minimization Procedure
and its Application to Outcome Weighted Learning [5.025486694392673]
We propose the first differentially private wERM algorithm, backed by a rigorous theoretical proof of its DP guarantees.
We evaluate the performance of the DP-wERM application to weighted learning (OWL) in a simulation study and in a real clinical trial of melatonin for sleep health.
arXiv Detail & Related papers (2023-07-24T21:03:25Z) - Amplitude-Varying Perturbation for Balancing Privacy and Utility in
Federated Learning [86.08285033925597]
This paper presents a new DP perturbation mechanism with a time-varying noise amplitude to protect the privacy of federated learning.
We derive an online refinement of the series to prevent FL from premature convergence resulting from excessive perturbation noise.
The contribution of the new DP mechanism to the convergence and accuracy of privacy-preserving FL is corroborated, compared to the state-of-the-art Gaussian noise mechanism with a persistent noise amplitude.
arXiv Detail & Related papers (2023-03-07T22:52:40Z) - On the Privacy-Robustness-Utility Trilemma in Distributed Learning [7.778461949427662]
We present the first tight analysis of the error incurred by any algorithm ensuring robustness against a fraction of adversarial machines.
Our analysis exhibits a fundamental trade-off between privacy, robustness, and utility.
arXiv Detail & Related papers (2023-02-09T17:24:18Z) - A Differentially Private Framework for Deep Learning with Convexified
Loss Functions [4.059849656394191]
Differential privacy (DP) has been applied in deep learning for preserving privacy of the underlying training sets.
Existing DP practice falls into three categories - objective perturbation, gradient perturbation and output perturbation.
We propose a novel output perturbation framework by injecting DP noise into a randomly sampled neuron.
arXiv Detail & Related papers (2022-04-03T11:10:05Z) - Sensitivity analysis in differentially private machine learning using
hybrid automatic differentiation [54.88777449903538]
We introduce a novel textithybrid automatic differentiation (AD) system for sensitivity analysis.
This enables modelling the sensitivity of arbitrary differentiable function compositions, such as the training of neural networks on private data.
Our approach can enable the principled reasoning about privacy loss in the setting of data processing.
arXiv Detail & Related papers (2021-07-09T07:19:23Z) - Smoothed Differential Privacy [55.415581832037084]
Differential privacy (DP) is a widely-accepted and widely-applied notion of privacy based on worst-case analysis.
In this paper, we propose a natural extension of DP following the worst average-case idea behind the celebrated smoothed analysis.
We prove that any discrete mechanism with sampling procedures is more private than what DP predicts, while many continuous mechanisms with sampling procedures are still non-private under smoothed DP.
arXiv Detail & Related papers (2021-07-04T06:55:45Z) - Accuracy, Interpretability, and Differential Privacy via Explainable
Boosting [22.30100748652558]
We show that adding differential privacy to Explainable Boosting Machines (EBMs) yields state-of-the-art accuracy while protecting privacy.
Our experiments on multiple classification and regression datasets show that DP-EBM models suffer surprisingly little accuracy loss even with strong differential privacy guarantees.
arXiv Detail & Related papers (2021-06-17T17:33:00Z) - On the Practicality of Differential Privacy in Federated Learning by
Tuning Iteration Times [51.61278695776151]
Federated Learning (FL) is well known for its privacy protection when training machine learning models among distributed clients collaboratively.
Recent studies have pointed out that the naive FL is susceptible to gradient leakage attacks.
Differential Privacy (DP) emerges as a promising countermeasure to defend against gradient leakage attacks.
arXiv Detail & Related papers (2021-01-11T19:43:12Z) - Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users.
An adversary may still be able to infer the private training data by attacking the released model.
Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
arXiv Detail & Related papers (2020-05-01T04:28:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.