Correlated Privacy Mechanisms for Differentially Private Distributed Mean Estimation

• URL: http://arxiv.org/abs/2407.03289v1
• Date: Wed, 3 Jul 2024 17:22:33 GMT
• Title: Correlated Privacy Mechanisms for Differentially Private Distributed Mean Estimation
• Authors: Sajani Vithana, Viveck R. Cadambe, Flavio P. Calmon, Haewon Jeong,
• Abstract summary: CorDP-DME is a novel DP-DME that spans the gap between local differential privacy (LDP) and distributed DP (SecAgg) We provide an information-theoretic analysis of CorDP-DME, and derive theoretical guarantees for utility under any given privacy parameters.
• Score: 8.660393575612169
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Differentially private distributed mean estimation (DP-DME) is a fundamental building block in privacy-preserving federated learning, where a central server estimates the mean of $d$-dimensional vectors held by $n$ users while ensuring $(\epsilon,\delta)$-DP. Local differential privacy (LDP) and distributed DP with secure aggregation (SecAgg) are the most common notions of DP used in DP-DME settings with an untrusted server. LDP provides strong resilience to dropouts, colluding users, and malicious server attacks, but suffers from poor utility. In contrast, SecAgg-based DP-DME achieves an $O(n)$ utility gain over LDP in DME, but requires increased communication and computation overheads and complex multi-round protocols to handle dropouts and malicious attacks. In this work, we propose CorDP-DME, a novel DP-DME mechanism that spans the gap between DME with LDP and distributed DP, offering a favorable balance between utility and resilience to dropout and collusion. CorDP-DME is based on correlated Gaussian noise, ensuring DP without the perfect conditional privacy guarantees of SecAgg-based approaches. We provide an information-theoretic analysis of CorDP-DME, and derive theoretical guarantees for utility under any given privacy parameters and dropout/colluding user thresholds. Our results demonstrate that (anti) correlated Gaussian DP mechanisms can significantly improve utility in mean estimation tasks compared to LDP -- even in adversarial settings -- while maintaining better resilience to dropouts and attacks compared to distributed DP.

Related papers

• Secure Stateful Aggregation: A Practical Protocol with Applications in Differentially-Private Federated Learning [36.42916779389165]
  DP-FTRL based approaches have already seen widespread deployment in industry. We introduce secure stateful aggregation: a simple append-only data structure that allows for the private storage of aggregate values. We observe that secure stateful aggregation suffices for realizing DP-FTRL-based private federated learning.
  arXiv  Detail & Related papers  (2024-10-15T07:45:18Z)
• FastLloyd: Federated, Accurate, Secure, and Tunable $k$-Means Clustering with Differential Privacy [26.927356987142407]
  We study the problem of privacy-preserving $k$-means clustering in the horizontally federated setting. Existing approaches using secure computation suffer from substantial overheads and do not offer output privacy. By utilizing the computational DP model, we design a lightweight, secure aggregation-based approach that achieves four orders of magnitude speed-up.
  arXiv  Detail & Related papers  (2024-05-03T19:04:37Z)
• Improved Communication-Privacy Trade-offs in $L_2$ Mean Estimation under Streaming Differential Privacy [47.997934291881414]
  Existing mean estimation schemes are usually optimized for $L_infty$ geometry and rely on random rotation or Kashin's representation to adapt to $L$ geometry. We introduce a novel privacy accounting method for the sparsified Gaussian mechanism that incorporates the randomness inherent in sparsification into the DP. Unlike previous approaches, our accounting algorithm directly operates in $L$ geometry, yielding MSEs that fast converge to those of the Gaussian mechanism.
  arXiv  Detail & Related papers  (2024-05-02T03:48:47Z)
• Privacy Amplification for the Gaussian Mechanism via Bounded Support [64.86780616066575]
  Data-dependent privacy accounting frameworks such as per-instance differential privacy (pDP) and Fisher information loss (FIL) confer fine-grained privacy guarantees for individuals in a fixed training dataset. We propose simple modifications of the Gaussian mechanism with bounded support, showing that they amplify privacy guarantees under data-dependent accounting.
  arXiv  Detail & Related papers  (2024-03-07T21:22:07Z)
• Conciliating Privacy and Utility in Data Releases via Individual Differential Privacy and Microaggregation [4.287502453001108]
  $epsilon$-Differential privacy (DP) is a well-known privacy model that offers strong privacy guarantees. We propose $epsilon$-individual differential privacy (iDP), which causes less data distortion while providing the same protection as DP to subjects. We report on experiments that show how our approach can provide strong privacy (small $epsilon$) while yielding protected data that do not significantly degrade the accuracy of secondary data analysis.
  arXiv  Detail & Related papers  (2023-12-21T10:23:18Z)
• Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach [62.000948039914135]
  Using Differentially Private Gradient Descent with Gradient Clipping (DPSGD-GC) to ensure Differential Privacy (DP) comes at the cost of model performance degradation. We propose a new error-feedback (EF) DP algorithm as an alternative to DPSGD-GC. We establish an algorithm-specific DP analysis for our proposed algorithm, providing privacy guarantees based on R'enyi DP.
  arXiv  Detail & Related papers  (2023-11-24T17:56:44Z)
• Connect the Dots: Tighter Discrete Approximations of Privacy Loss Distributions [49.726408540784334]
  Key question in PLD-based accounting is how to approximate any (potentially continuous) PLD with a PLD over any specified discrete support. We show that our pessimistic estimate is the best possible among all pessimistic estimates.
  arXiv  Detail & Related papers  (2022-07-10T04:25:02Z)
• Differentially Private Regret Minimization in Episodic Markov Decision Processes [6.396288020763144]
  We study regret in finite horizon tabular Markov decision processes (MDPs) under the constraints of differential privacy (DP) This is motivated by the widespread applications of reinforcement learning (RL) in real-world sequential decision making problems.
  arXiv  Detail & Related papers  (2021-12-20T15:12:23Z)
• Differentially Private Federated Bayesian Optimization with Distributed Exploration [48.9049546219643]
  We introduce differential privacy (DP) into the training of deep neural networks through a general framework for adding DP to iterative algorithms. We show that DP-FTS-DE achieves high utility (competitive performance) with a strong privacy guarantee. We also use real-world experiments to show that DP-FTS-DE induces a trade-off between privacy and utility.
  arXiv  Detail & Related papers  (2021-10-27T04:11:06Z)
• Three Variants of Differential Privacy: Lossless Conversion and Applications [13.057076084452016]
  We consider three different variants of differential privacy (DP), namely approximate DP, R'enyi RDP, and hypothesis test. In the first part, we develop a machinery for relating approximate DP to iterations based on the joint range of two $f$-divergences. As an application, we apply our result to the moments framework for characterizing privacy guarantees of noisy gradient descent.
  arXiv  Detail & Related papers  (2020-08-14T18:23:50Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.