Private Sketches for Linear Regression

• URL: http://arxiv.org/abs/2511.07365v1
• Date: Mon, 10 Nov 2025 18:22:40 GMT
• Title: Private Sketches for Linear Regression
• Authors: Shrutimoy Das, Debanuj Nayak, Anirban Dasgupta,
• Abstract summary: We release differentially private sketches for the problems of least squares and absolute deviations regression.<n>The availability of these private sketches facilitates the application of commonly available solvers for regression.
• Score: 2.959915525479836
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Linear regression is frequently applied in a variety of domains. In order to improve the efficiency of these methods, various methods have been developed that compute summaries or \emph{sketches} of the datasets. Certain domains, however, contain sensitive data which necessitates that the application of these statistical methods does not reveal private information. Differentially private (DP) linear regression methods have been developed for mitigating this problem. These techniques typically involve estimating a noisy version of the parameter vector. Instead, we propose releasing private sketches of the datasets. We present differentially private sketches for the problems of least squares regression, as well as least absolute deviations regression. The availability of these private sketches facilitates the application of commonly available solvers for regression, without the risk of privacy leakage.

Related papers

• Linear-Time User-Level DP-SCO via Robust Statistics [55.350093142673316]
  User-level differentially private convex optimization (DP-SCO) has garnered significant attention due to the importance of safeguarding user privacy in machine learning applications.<n>Current methods, such as those based on differentially private gradient descent (DP-SGD), often struggle with high noise accumulation and suboptimal utility.<n>We introduce a novel linear-time algorithm that leverages robust statistics, specifically the median and trimmed mean, to overcome these challenges.
  arXiv  Detail & Related papers  (2025-02-13T02:05:45Z)
• Private Linear Regression with Differential Privacy and PAC Privacy [0.0]
  Most existing privacy-preserving linear regression methods rely on the well-established framework of differential privacy.<n> PAC Privacy has not yet been explored in this context.<n>We compare linear regression models trained with differential privacy and PAC privacy across three real-world datasets.
  arXiv  Detail & Related papers  (2024-12-03T17:04:14Z)
• Differential Privacy Regularization: Protecting Training Data Through Loss Function Regularization [49.1574468325115]
  Training machine learning models based on neural networks requires large datasets, which may contain sensitive information. Differentially private SGD [DP-SGD] requires the modification of the standard gradient descent [SGD] algorithm for training new models. A novel regularization strategy is proposed to achieve the same goal in a more efficient manner.
  arXiv  Detail & Related papers  (2024-09-25T17:59:32Z)
• Differentially Private Linear Regression with Linked Data [3.9325957466009203]
  Differential privacy, a mathematical notion from computer science, is a rising tool offering robust privacy guarantees. Recent work focuses on developing differentially private versions of individual statistical and machine learning tasks. We present two differentially private algorithms for linear regression with linked data.
  arXiv  Detail & Related papers  (2023-08-01T21:00:19Z)
• Differentially Private Statistical Inference through $\eta$-Divergence One Posterior Sampling [2.8544822698499255]
  We propose a posterior sampling scheme from a generalised posterior targeting the minimisation of the $beta$-divergence between the model and the data generating process. This provides private estimation that is generally applicable without requiring changes to the underlying model. We show that $beta$D-Bayes produces more precise inference estimation for the same privacy guarantees.
  arXiv  Detail & Related papers  (2023-07-11T12:00:15Z)
• Better Private Linear Regression Through Better Private Feature Selection [18.884088349732973]
  We introduce a differentially private feature selection method based on Kendall rank correlation. We prove a utility guarantee for the setting where features are normally distributed. We find that adding this private feature selection step before regression significantly broadens the applicability of plug-and-play'' private linear regression algorithms.
  arXiv  Detail & Related papers  (2023-06-01T17:21:10Z)
• Improved Differentially Private Regression via Gradient Boosting [38.09948758699131]
  We give a new algorithm for private linear regression based on gradient boosting. In addition to a comprehensive set of experiments, we give theoretical insights to explain this behavior.
  arXiv  Detail & Related papers  (2023-03-06T19:20:52Z)
• Near-optimal inference in adaptive linear regression [60.08422051718195]
  Even simple methods like least squares can exhibit non-normal behavior when data is collected in an adaptive manner. We propose a family of online debiasing estimators to correct these distributional anomalies in at least squares estimation. We demonstrate the usefulness of our theory via applications to multi-armed bandit, autoregressive time series estimation, and active learning with exploration.
  arXiv  Detail & Related papers  (2021-07-05T21:05:11Z)
• Near-Optimal Linear Regression under Distribution Shift [63.87137348308034]
  We show that linear minimax estimators are within an absolute constant of the minimax risk even among nonlinear estimators for various source/target distributions.
  arXiv  Detail & Related papers  (2021-06-23T00:52:50Z)
• Learning Probabilistic Ordinal Embeddings for Uncertainty-Aware Regression [91.3373131262391]
  Uncertainty is the only certainty there is. Traditionally, the direct regression formulation is considered and the uncertainty is modeled by modifying the output space to a certain family of probabilistic distributions. How to model the uncertainty within the present-day technologies for regression remains an open issue.
  arXiv  Detail & Related papers  (2021-03-25T06:56:09Z)
• Benign Overfitting of Constant-Stepsize SGD for Linear Regression [122.70478935214128]
  inductive biases are central in preventing overfitting empirically. This work considers this issue in arguably the most basic setting: constant-stepsize SGD for linear regression. We reflect on a number of notable differences between the algorithmic regularization afforded by (unregularized) SGD in comparison to ordinary least squares.
  arXiv  Detail & Related papers  (2021-03-23T17:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.