Differentially Private Estimation via Statistical Depth

• URL: http://arxiv.org/abs/2207.12602v1
• Date: Tue, 26 Jul 2022 01:59:07 GMT
• Title: Differentially Private Estimation via Statistical Depth
• Authors: Ryan Cumings-Menon
• Abstract summary: Two notions of statistical depth are used to motivate new approximate DP location and regression estimators. To avoid requiring that users specify a priori bounds on the estimates and/or the observations, variants of these DP mechanisms are described.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Constructing a differentially private (DP) estimator requires deriving the maximum influence of an observation, which can be difficult in the absence of exogenous bounds on the input data or the estimator, especially in high dimensional settings. This paper shows that standard notions of statistical depth, i.e., halfspace depth and regression depth, are particularly advantageous in this regard, both in the sense that the maximum influence of a single observation is easy to analyze and that this value is typically low. This is used to motivate new approximate DP location and regression estimators using the maximizers of these two notions of statistical depth. A more computationally efficient variant of the approximate DP regression estimator is also provided. Also, to avoid requiring that users specify a priori bounds on the estimates and/or the observations, variants of these DP mechanisms are described that satisfy random differential privacy (RDP), which is a relaxation of differential privacy provided by Hall, Wasserman, and Rinaldo (2013). We also provide simulations of the two DP regression methods proposed here. The proposed estimators appear to perform favorably relative to the existing DP regression methods we consider in these simulations when either the sample size is at least 100-200 or the privacy-loss budget is sufficiently high.

Related papers

• High-Dimensional Differentially Private Quantile Regression: Distributed Estimation and Statistical Inference [0.26784722398800515]
  We propose a differentially private quantile regression method for high-dimensional data in a distributed setting.<n>We develop a differentially private estimation algorithm with iterative updates, ensuring near-optimal statistical accuracy and formal privacy guarantees.
  arXiv  Detail & Related papers  (2025-08-07T09:47:44Z)
• Statistical Inference for Differentially Private Stochastic Gradient Descent [14.360996967498002]
  This paper bridges the gap between existing statistical methods and Differentially Private Gradient Descent (DP-SGD)<n>For the output of DP-SGD, we show that the variance decomposes into statistical, sampling, and privacy-induced components.<n>Two methods are proposed for constructing valid confidence intervals: the plug-in method and the random scaling method.
  arXiv  Detail & Related papers  (2025-07-28T06:45:15Z)
• Better Rates for Private Linear Regression in the Proportional Regime via Aggressive Clipping [19.186034457189162]
  A common approach is to set the clipping constant much larger than the expected norm of the per-sample gradients.<n>While simplifying the analysis, this is however in sharp contrast with what empirical evidence suggests to optimize performance.<n>Our work bridges this gap between theory and practice by crucially operating in a regime where clipping happens frequently.
  arXiv  Detail & Related papers  (2025-05-22T07:34:27Z)
• Private Statistical Estimation via Truncation [2.3910125679710665]
  We introduce a novel framework for differentially private statistical estimation via data truncation, addressing a key challenge in DP estimation when the data support is unbounded.<n>By leveraging techniques from truncated statistics, we develop computationally efficient DP estimators for exponential family distributions.
  arXiv  Detail & Related papers  (2025-05-18T20:38:38Z)
• Tukey Depth Mechanisms for Practical Private Mean Estimation [5.298789668079187]
  Mean estimation is a fundamental task in statistics and a focus within differentially private statistical estimation. In this work, we take the first step to bridge the gap by implementing the (Restricted) Tukey Depth Mechanism. Our implementations enable the use of these mechanisms for small sample sizes or low-dimensional data.
  arXiv  Detail & Related papers  (2025-02-25T23:21:54Z)
• Deep Partially Linear Transformation Model for Right-Censored Survival Data [9.991327369572819]
  This paper introduces a deep partially linear transformation model (DPLTM) as a general and flexible framework for estimation, inference and prediction. Comprehensive simulation studies demonstrate the impressive performance of the proposed estimation procedure in terms of both estimation accuracy and prediction power.
  arXiv  Detail & Related papers  (2024-12-10T15:50:43Z)
• Distributionally Robust Optimization as a Scalable Framework to Characterize Extreme Value Distributions [22.765095010254118]
  The goal of this paper is to develop distributionally robust optimization (DRO) estimators, specifically for multidimensional Extreme Value Theory (EVT) statistics. In order to mitigate over-conservative estimates while enhancing out-of-sample performance, we study DRO estimators informed by semi-parametric max-stable constraints in the space of point processes. Both approaches are validated using synthetically generated data, recovering prescribed characteristics, and verifying the efficacy of the proposed techniques.
  arXiv  Detail & Related papers  (2024-07-31T19:45:27Z)
• Differentially Private Log-Location-Scale Regression Using Functional Mechanism [1.2277343096128712]
  This article introduces differentially private log-location-scale (DP-LLS) regression models, which incorporate differential privacy into LLS regression. We will derive the sensitivities utilized to determine the magnitude of the injected noise and prove that the proposed DP-LLS models satisfy $epsilon$-differential privacy.
  arXiv  Detail & Related papers  (2024-04-12T04:14:08Z)
• A Mean Field Approach to Empirical Bayes Estimation in High-dimensional Linear Regression [8.345523969593492]
  We study empirical Bayes estimation in high-dimensional linear regression. We adopt a variational empirical Bayes approach, introduced originally in Carbonetto and Stephens (2012) and Kim et al. (2022). This provides the first rigorous empirical Bayes method in a high-dimensional regression setting without sparsity.
  arXiv  Detail & Related papers  (2023-09-28T20:51:40Z)
• Bias-Aware Minimisation: Understanding and Mitigating Estimator Bias in Private SGD [56.01810892677744]
  We show a connection between per-sample gradient norms and the estimation bias of the private gradient oracle used in DP-SGD. We propose Bias-Aware Minimisation (BAM) that allows for the provable reduction of private gradient estimator bias.
  arXiv  Detail & Related papers  (2023-08-23T09:20:41Z)
• Robust Bayesian Subspace Identification for Small Data Sets [91.3755431537592]
  We propose regularized estimators, shrinkage estimators and Bayesian estimation to reduce the effect of variance. Our experimental results show that our proposed estimators may reduce the estimation risk up to $40%$ of that of traditional subspace methods.
  arXiv  Detail & Related papers  (2022-12-29T00:29:04Z)
• Differentially private multivariate medians [4.588028371034407]
  We develop novel finite-sample performance guarantees for differentially private depth-based medians. We show that under Cauchy marginals, the cost of heavy-tailed location estimation outweighs the cost of privacy.
  arXiv  Detail & Related papers  (2022-10-12T17:56:04Z)
• Normalized/Clipped SGD with Perturbation for Differentially Private Non-Convex Optimization [94.06564567766475]
  DP-SGD and DP-NSGD mitigate the risk of large models memorizing sensitive training data. We show that these two algorithms achieve similar best accuracy while DP-NSGD is comparatively easier to tune than DP-SGD.
  arXiv  Detail & Related papers  (2022-06-27T03:45:02Z)
• On Variance Estimation of Random Forests [0.0]
  This paper develops an unbiased variance estimator based on incomplete U-statistics. We show that our estimators enjoy lower bias and more accurate confidence interval coverage without additional computational costs.
  arXiv  Detail & Related papers  (2022-02-18T03:35:47Z)
• Private Stochastic Non-Convex Optimization: Adaptive Algorithms and Tighter Generalization Bounds [72.63031036770425]
  We propose differentially private (DP) algorithms for bound non-dimensional optimization. We demonstrate two popular deep learning methods on the empirical advantages over standard gradient methods.
  arXiv  Detail & Related papers  (2020-06-24T06:01:24Z)
• On Projection Robust Optimal Transport: Sample Complexity and Model Misspecification [101.0377583883137]
  Projection robust (PR) OT seeks to maximize the OT cost between two measures by choosing a $k$-dimensional subspace onto which they can be projected. Our first contribution is to establish several fundamental statistical properties of PR Wasserstein distances. Next, we propose the integral PR Wasserstein (IPRW) distance as an alternative to the PRW distance, by averaging rather than optimizing on subspaces.
  arXiv  Detail & Related papers  (2020-06-22T14:35:33Z)
• Nonparametric Estimation of the Fisher Information and Its Applications [82.00720226775964]
  This paper considers the problem of estimation of the Fisher information for location from a random sample of size $n$. An estimator proposed by Bhattacharya is revisited and improved convergence rates are derived. A new estimator, termed a clipped estimator, is proposed.
  arXiv  Detail & Related papers  (2020-05-07T17:21:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.