Differentially private and decentralized randomized power method

• URL: http://arxiv.org/abs/2411.01931v2
• Date: Tue, 26 Nov 2024 12:06:12 GMT
• Title: Differentially private and decentralized randomized power method
• Authors: Julien Nicolas, César Sabater, Mohamed Maouche, Sonia Ben Mokhtar, Mark Coates,
• Abstract summary: We propose a strategy to reduce the variance of the noise introduced to achieve Differential Privacy (DP) We adapt the method to a decentralized framework with a low computational and communication overhead, while preserving the accuracy. We show that it is possible to use a noise scale in the decentralized setting that is similar to the one in the centralized setting.
• Score: 15.955127242261808
• License:
• Abstract: The randomized power method has gained significant interest due to its simplicity and efficient handling of large-scale spectral analysis and recommendation tasks. As modern datasets contain sensitive private information, we need to give formal guarantees on the possible privacy leaks caused by this method. This paper focuses on enhancing privacy preserving variants of the method. We propose a strategy to reduce the variance of the noise introduced to achieve Differential Privacy (DP). We also adapt the method to a decentralized framework with a low computational and communication overhead, while preserving the accuracy. We leverage Secure Aggregation (a form of Multi-Party Computation) to allow the algorithm to perform computations using data distributed among multiple users or devices, without revealing individual data. We show that it is possible to use a noise scale in the decentralized setting that is similar to the one in the centralized setting. We improve upon existing convergence bounds for both the centralized and decentralized versions. The proposed method is especially relevant for decentralized applications such as distributed recommender systems, where privacy concerns are paramount.

Related papers

• Differentially Private Random Feature Model [52.468511541184895]
  We produce a differentially private random feature model for privacy-preserving kernel machines. We show that our method preserves privacy and derive a generalization error bound for the method.
  arXiv  Detail & Related papers  (2024-12-06T05:31:08Z)
• Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learning [47.042811490685324]
  Mitigating the risk of this information leakage, using state of the art differentially private algorithms, also does not come for free. In this paper, we consider a representation learning objective that various parties collaboratively refine on a federated model, with differential privacy guarantees. We observe a significant performance improvement over the prior work under the same small privacy budget.
  arXiv  Detail & Related papers  (2023-09-11T14:46:55Z)
• Samplable Anonymous Aggregation for Private Federated Data Analysis [25.35309084903802]
  Locally differentially private algorithms require little trust but are (provably) limited in their utility. Centrally differentially private algorithms can allow significantly better utility but require a trusted curator. Our first contribution is to propose a new primitive that allows for efficient implementation of several commonly used algorithms.
  arXiv  Detail & Related papers  (2023-07-27T17:19:37Z)
• Breaking the Communication-Privacy-Accuracy Tradeoff with $f$-Differential Privacy [51.11280118806893]
  We consider a federated data analytics problem in which a server coordinates the collaborative data analysis of multiple users with privacy concerns and limited communication capability. We study the local differential privacy guarantees of discrete-valued mechanisms with finite output space through the lens of $f$-differential privacy (DP) More specifically, we advance the existing literature by deriving tight $f$-DP guarantees for a variety of discrete-valued mechanisms.
  arXiv  Detail & Related papers  (2023-02-19T16:58:53Z)
• On Differential Privacy for Federated Learning in Wireless Systems with Multiple Base Stations [90.53293906751747]
  We consider a federated learning model in a wireless system with multiple base stations and inter-cell interference. We show the convergence behavior of the learning process by deriving an upper bound on its optimality gap. Our proposed scheduler improves the average accuracy of the predictions compared with a random scheduler.
  arXiv  Detail & Related papers  (2022-08-25T03:37:11Z)
• Differentially Private Vertical Federated Clustering [13.27934054846057]
  In many applications, multiple parties have private data regarding the same set of users but on disjoint sets of attributes. To enable model learning while protecting the privacy of the data subjects, we need vertical federated learning (VFL) techniques. The algorithm proposed in this paper is the first practical solution for differentially private vertical federated k-means clustering.
  arXiv  Detail & Related papers  (2022-08-02T19:23:48Z)
• Decentralized Stochastic Optimization with Inherent Privacy Protection [103.62463469366557]
  Decentralized optimization is the basic building block of modern collaborative machine learning, distributed estimation and control, and large-scale sensing. Since involved data, privacy protection has become an increasingly pressing need in the implementation of decentralized optimization algorithms.
  arXiv  Detail & Related papers  (2022-05-08T14:38:23Z)
• Network Shuffling: Privacy Amplification via Random Walks [21.685747588753514]
  We introduce network shuffling, a decentralized mechanism where users exchange data in a random-walk fashion on a network/graph. We show that the privacy amplification rate is similar to other privacy amplification techniques such as uniform shuffling.
  arXiv  Detail & Related papers  (2022-04-08T08:36:06Z)
• Graph-Homomorphic Perturbations for Private Decentralized Learning [64.26238893241322]
  Local exchange of estimates allows inference of data based on private data. perturbations chosen independently at every agent, resulting in a significant performance loss. We propose an alternative scheme, which constructs perturbations according to a particular nullspace condition, allowing them to be invisible.
  arXiv  Detail & Related papers  (2020-10-23T10:35:35Z)
• Privacy Amplification via Random Check-Ins [38.72327434015975]
  Differentially Private Gradient Descent (DP-SGD) forms a fundamental building block in many applications for learning over sensitive data. In this paper, we focus on conducting iterative methods like DP-SGD in the setting of federated learning (FL) wherein the data is distributed among many devices (clients) Our main contribution is the emphrandom check-in distributed protocol, which crucially relies only on randomized participation decisions made locally and independently by each client.
  arXiv  Detail & Related papers  (2020-07-13T18:14:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.