Differentially Private Bootstrap: New Privacy Analysis and Inference
Strategies
- URL: http://arxiv.org/abs/2210.06140v2
- Date: Fri, 21 Apr 2023 13:12:44 GMT
- Title: Differentially Private Bootstrap: New Privacy Analysis and Inference
Strategies
- Authors: Zhanyu Wang, Guang Cheng, Jordan Awan
- Abstract summary: Differentially private (DP) mechanisms protect individual-level information by introducing randomness into the statistical analysis procedure.
We examine a DP bootstrap procedure that releases multiple private bootstrap estimates to infer the sampling distribution and construct confidence intervals (CIs)
Our privacy analysis presents new results on the privacy cost of a single DP bootstrap estimate, applicable to any DP mechanisms, and identifies some misapplications of the bootstrap in the existing literature.
- Score: 28.95350475681164
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Differentially private (DP) mechanisms protect individual-level information
by introducing randomness into the statistical analysis procedure. Despite the
availability of numerous DP tools, there remains a lack of general techniques
for conducting statistical inference under DP. We examine a DP bootstrap
procedure that releases multiple private bootstrap estimates to infer the
sampling distribution and construct confidence intervals (CIs). Our privacy
analysis presents new results on the privacy cost of a single DP bootstrap
estimate, applicable to any DP mechanisms, and identifies some misapplications
of the bootstrap in the existing literature. Using the Gaussian-DP (GDP)
framework (Dong et al.,2022), we show that the release of $B$ DP bootstrap
estimates from mechanisms satisfying $(\mu/\sqrt{(2-2/\mathrm{e})B})$-GDP
asymptotically satisfies $\mu$-GDP as $B$ goes to infinity. Moreover, we use
deconvolution with the DP bootstrap estimates to accurately infer the sampling
distribution, which is novel in DP. We derive CIs from our density estimate for
tasks such as population mean estimation, logistic regression, and quantile
regression, and we compare them to existing methods using simulations and
real-world experiments on 2016 Canada Census data. Our private CIs achieve the
nominal coverage level and offer the first approach to private inference for
quantile regression.
Related papers
- Private Mean Estimation with Person-Level Differential Privacy [6.621676316292624]
We study person-level differentially private mean estimation in the case where each person holds multiple samples.
We give computationally efficient algorithms under approximate-DP and computationally inefficient algorithms under pure DP, and our nearly matching lower bounds hold for the most permissive case of approximate DP.
arXiv Detail & Related papers (2024-05-30T18:20:35Z) - How Private are DP-SGD Implementations? [61.19794019914523]
We show that there can be a substantial gap between the privacy analysis when using the two types of batch sampling.
Our result shows that there can be a substantial gap between the privacy analysis when using the two types of batch sampling.
arXiv Detail & Related papers (2024-03-26T13:02:43Z) - Resampling methods for private statistical inference [1.8110941972682346]
We consider the task of constructing confidence intervals with differential privacy.
We propose two private variants of the non-parametric bootstrap, which privately compute the median of the results of multiple "little" bootstraps run on partitions of the data.
For a fixed differential privacy parameter $epsilon$, our methods enjoy the same error rates as that of the non-private bootstrap to within logarithmic factors in the sample size $n$.
arXiv Detail & Related papers (2024-02-11T08:59:02Z) - Differentially Private Statistical Inference through $\beta$-Divergence
One Posterior Sampling [2.8544822698499255]
We propose a posterior sampling scheme from a generalised posterior targeting the minimisation of the $beta$-divergence between the model and the data generating process.
This provides private estimation that is generally applicable without requiring changes to the underlying model.
We show that $beta$D-Bayes produces more precise inference estimation for the same privacy guarantees.
arXiv Detail & Related papers (2023-07-11T12:00:15Z) - Recycling Scraps: Improving Private Learning by Leveraging Intermediate
Checkpoints [17.654346227497403]
This work explores various methods that aggregate intermediate checkpoints to improve the utility of DP training.
We show that checkpoint aggregations provide significant gains in the prediction accuracy over the existing SOTA for CIFAR10 and StackOverflow datasets.
Finally, we show that the sample variance from last few checkpoints provides a good approximation of the variance of the final model of a DP run.
arXiv Detail & Related papers (2022-10-04T19:21:00Z) - Differentially Private Estimation via Statistical Depth [0.0]
Two notions of statistical depth are used to motivate new approximate DP location and regression estimators.
To avoid requiring that users specify a priori bounds on the estimates and/or the observations, variants of these DP mechanisms are described.
arXiv Detail & Related papers (2022-07-26T01:59:07Z) - Normalized/Clipped SGD with Perturbation for Differentially Private
Non-Convex Optimization [94.06564567766475]
DP-SGD and DP-NSGD mitigate the risk of large models memorizing sensitive training data.
We show that these two algorithms achieve similar best accuracy while DP-NSGD is comparatively easier to tune than DP-SGD.
arXiv Detail & Related papers (2022-06-27T03:45:02Z) - Optimal Membership Inference Bounds for Adaptive Composition of Sampled
Gaussian Mechanisms [93.44378960676897]
Given a trained model and a data sample, membership-inference (MI) attacks predict whether the sample was in the model's training set.
A common countermeasure against MI attacks is to utilize differential privacy (DP) during model training to mask the presence of individual examples.
In this paper, we derive bounds for the textitadvantage of an adversary mounting a MI attack, and demonstrate tightness for the widely-used Gaussian mechanism.
arXiv Detail & Related papers (2022-04-12T22:36:56Z) - Nonparametric extensions of randomized response for private confidence sets [51.75485869914048]
This work derives methods for performing nonparametric, nonasymptotic statistical inference for population means under the constraint of local differential privacy (LDP)
We present confidence intervals (CI) and time-uniform confidence sequences (CS) for $mustar$ when only given access to the privatized data.
arXiv Detail & Related papers (2022-02-17T16:04:49Z) - On the Practicality of Differential Privacy in Federated Learning by
Tuning Iteration Times [51.61278695776151]
Federated Learning (FL) is well known for its privacy protection when training machine learning models among distributed clients collaboratively.
Recent studies have pointed out that the naive FL is susceptible to gradient leakage attacks.
Differential Privacy (DP) emerges as a promising countermeasure to defend against gradient leakage attacks.
arXiv Detail & Related papers (2021-01-11T19:43:12Z) - Private Stochastic Non-Convex Optimization: Adaptive Algorithms and
Tighter Generalization Bounds [72.63031036770425]
We propose differentially private (DP) algorithms for bound non-dimensional optimization.
We demonstrate two popular deep learning methods on the empirical advantages over standard gradient methods.
arXiv Detail & Related papers (2020-06-24T06:01:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.