Analyzing the Differentially Private Theil-Sen Estimator for Simple Linear Regression

• URL: http://arxiv.org/abs/2207.13289v2
• Date: Mon, 11 Mar 2024 23:28:06 GMT
• Title: Analyzing the Differentially Private Theil-Sen Estimator for Simple Linear Regression
• Authors: Jayshree Sarathy, Salil Vadhan,
• Abstract summary: We provide a rigorous, finite-sample analysis of DPTheilSen's privacy and accuracy properties. We show how to produce differentially private confidence intervals to accompany its point estimates.
• Score: 0.9208007322096533
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we study differentially private point and confidence interval estimators for simple linear regression. Motivated by recent work that highlights the strong empirical performance of an algorithm based on robust statistics, DPTheilSen, we provide a rigorous, finite-sample analysis of its privacy and accuracy properties, offer guidance on setting hyperparameters, and show how to produce differentially private confidence intervals to accompany its point estimates.

Related papers

• Statistical Inference for Temporal Difference Learning with Linear Function Approximation [62.69448336714418]
  Temporal Difference (TD) learning, arguably the most widely used for policy evaluation, serves as a natural framework for this purpose. In this paper, we study the consistency properties of TD learning with Polyak-Ruppert averaging and linear function approximation, and obtain three significant improvements over existing results.
  arXiv  Detail & Related papers  (2024-10-21T15:34:44Z)
• Finite Sample Confidence Regions for Linear Regression Parameters Using Arbitrary Predictors [1.6860963320038902]
  We explore a novel methodology for constructing confidence regions for parameters of linear models, using predictions from any arbitrary predictor. The derived confidence regions can be cast as constraints within a Mixed Linear Programming framework, enabling optimisation of linear objectives. Unlike previous methods, the confidence region can be empty, which can be used for hypothesis testing.
  arXiv  Detail & Related papers  (2024-01-27T00:15:48Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• Differentially Private Confidence Intervals for Proportions under Stratified Random Sampling [14.066813980992132]
  With the increase of data privacy awareness, developing a private version of confidence intervals has gained growing attention. Recent work has been done around differentially private confidence intervals, yet rigorous methodologies on differentially private confidence intervals have not been studied. We propose three differentially private algorithms for constructing confidence intervals for proportions under stratified random sampling.
  arXiv  Detail & Related papers  (2023-01-19T21:25:41Z)
• Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalization [52.7137956951533]
  We argue that devising simpler methods for learning predictors on existing features is a promising direction for future research. We introduce Domain-Adjusted Regression (DARE), a convex objective for learning a linear predictor that is provably robust under a new model of distribution shift. Under a natural model, we prove that the DARE solution is the minimax-optimal predictor for a constrained set of test distributions.
  arXiv  Detail & Related papers  (2022-02-14T16:42:16Z)
• An automatic differentiation system for the age of differential privacy [65.35244647521989]
  Tritium is an automatic differentiation-based sensitivity analysis framework for differentially private (DP) machine learning (ML) We introduce Tritium, an automatic differentiation-based sensitivity analysis framework for differentially private (DP) machine learning (ML)
  arXiv  Detail & Related papers  (2021-09-22T08:07:42Z)
• Differentially private inference via noisy optimization [3.015622397986615]
  We show that robust statistics can be used in conjunction with noisy gradient descent or noisy Newton methods to obtain optimal private estimators. We demonstrate the effectiveness of a bias correction that leads to enhanced small-sample empirical performance in simulations.
  arXiv  Detail & Related papers  (2021-03-19T19:55:55Z)
• CoinDICE: Off-Policy Confidence Interval Estimation [107.86876722777535]
  We study high-confidence behavior-agnostic off-policy evaluation in reinforcement learning. We show in a variety of benchmarks that the confidence interval estimates are tighter and more accurate than existing methods.
  arXiv  Detail & Related papers  (2020-10-22T12:39:11Z)
• Parametric Bootstrap for Differentially Private Confidence Intervals [8.781431682774484]
  We develop a practical and general-purpose approach to construct confidence intervals for differentially private parametric estimation. We find that the parametric bootstrap is a simple and effective solution.
  arXiv  Detail & Related papers  (2020-06-14T00:08:19Z)
• GenDICE: Generalized Offline Estimation of Stationary Values [108.17309783125398]
  We show that effective estimation can still be achieved in important applications. Our approach is based on estimating a ratio that corrects for the discrepancy between the stationary and empirical distributions. The resulting algorithm, GenDICE, is straightforward and effective.
  arXiv  Detail & Related papers  (2020-02-21T00:27:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.