Efficient Node Selection in Private Personalized Decentralized Learning

• URL: http://arxiv.org/abs/2301.12755v2
• Date: Mon, 15 Jan 2024 15:52:46 GMT
• Title: Efficient Node Selection in Private Personalized Decentralized Learning
• Authors: Edvin Listo Zec, Johan \"Ostman, Olof Mogren, Daniel Gillblad
• Abstract summary: We propose Private Personalized Decentralized Learning (PPDL) to protect node privacy. PPDL combines secure aggregation and correlated adversarial multi-armed bandit optimization. We show that PPDL surpasses previous non-private methods in model performance on standard benchmarks.
• Score: 3.7784910521656654
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Personalized decentralized learning is a promising paradigm for distributed learning, enabling each node to train a local model on its own data and collaborate with other nodes to improve without sharing any data. However, this approach poses significant privacy risks, as nodes may inadvertently disclose sensitive information about their data or preferences through their collaboration choices. In this paper, we propose Private Personalized Decentralized Learning (PPDL), a novel approach that combines secure aggregation and correlated adversarial multi-armed bandit optimization to protect node privacy while facilitating efficient node selection. By leveraging dependencies between different arms, represented by potential collaborators, we demonstrate that PPDL can effectively identify suitable collaborators solely based on aggregated models. Additionally, we show that PPDL surpasses previous non-private methods in model performance on standard benchmarks under label and covariate shift scenarios.

Related papers

• Personalized Federated Learning for Cross-view Geo-localization [49.40531019551957]
  We propose a methodology combining Federated Learning (FL) with Cross-view Image Geo-localization (CVGL) techniques. Our method implements a coarse-to-fine approach, where clients share only the coarse feature extractors while keeping fine-grained features specific to local environments. Results demonstrate that our federated CVGL method achieves performance close to centralized training while maintaining data privacy.
  arXiv  Detail & Related papers  (2024-11-07T13:25:52Z)
• Robust Utility-Preserving Text Anonymization Based on Large Language Models [80.5266278002083]
  Text anonymization is crucial for sharing sensitive data while maintaining privacy. Existing techniques face the emerging challenges of re-identification attack ability of Large Language Models. This paper proposes a framework composed of three LLM-based components -- a privacy evaluator, a utility evaluator, and an optimization component.
  arXiv  Detail & Related papers  (2024-07-16T14:28:56Z)
• Decentralized Personalized Federated Learning [4.5836393132815045]
  We focus on creating a collaboration graph that guides each client in selecting suitable collaborators for training personalized models. Unlike traditional methods, our formulation identifies collaborators at a granular level by considering greedy relations of clients. We achieve this through a bi-level optimization framework that employs a constrained algorithm.
  arXiv  Detail & Related papers  (2024-06-10T17:58:48Z)
• Privacy Preserving Semi-Decentralized Mean Estimation over Intermittently-Connected Networks [59.43433767253956]
  We consider the problem of privately estimating the mean of vectors distributed across different nodes of an unreliable wireless network. In a semi-decentralized setup, nodes can collaborate with their neighbors to compute a local consensus, which they relay to a central server. We study the tradeoff between collaborative relaying and privacy leakage due to the data sharing among nodes.
  arXiv  Detail & Related papers  (2024-06-06T06:12:15Z)
• Approximate Gradient Coding for Privacy-Flexible Federated Learning with Non-IID Data [9.984630251008868]
  This work focuses on the challenges of non-IID data and stragglers/dropouts in federated learning. We introduce and explore a privacy-flexible paradigm that models parts of the clients' local data as non-private.
  arXiv  Detail & Related papers  (2024-04-04T15:29:50Z)
• Differentially Private Decentralized Learning with Random Walks [15.862152253607496]
  We characterize the privacy guarantees of decentralized learning with random walk algorithms, where a model is updated by traveling from one node to another along the edges of a communication graph. Our results reveal that random walk algorithms tends to yield better privacy guarantees than gossip algorithms for nodes close from each other.
  arXiv  Detail & Related papers  (2024-02-12T08:16:58Z)
• Independent Distribution Regularization for Private Graph Embedding [55.24441467292359]
  Graph embeddings are susceptible to attribute inference attacks, which allow attackers to infer private node attributes from the learned graph embeddings. To address these concerns, privacy-preserving graph embedding methods have emerged. We propose a novel approach called Private Variational Graph AutoEncoders (PVGAE) with the aid of independent distribution penalty as a regularization term.
  arXiv  Detail & Related papers  (2023-08-16T13:32:43Z)
• Generalizing Differentially Private Decentralized Deep Learning with Multi-Agent Consensus [11.414398732656839]
  We propose a framework that embeds differential privacy into decentralized deep learning and secures each agent's local dataset during and after cooperative training. We prove convergence guarantees for algorithms derived from this framework and demonstrate its practical utility when applied to subgradient and ADMM decentralized approaches.
  arXiv  Detail & Related papers  (2023-06-24T07:46:00Z)
• Personalized Graph Federated Learning with Differential Privacy [6.282767337715445]
  This paper presents a personalized graph federated learning (PGFL) framework in which distributedly connected servers and their respective edge devices collaboratively learn device or cluster-specific models. We study a variant of the PGFL implementation that utilizes differential privacy, specifically zero-concentrated differential privacy, where a noise sequence perturbs model exchanges. Our analysis shows that the algorithm ensures local differential privacy for all clients in terms of zero-concentrated differential privacy.
  arXiv  Detail & Related papers  (2023-06-10T09:52:01Z)
• Private Set Generation with Discriminative Information [63.851085173614]
  Differentially private data generation is a promising solution to the data privacy challenge. Existing private generative models are struggling with the utility of synthetic samples. We introduce a simple yet effective method that greatly improves the sample utility of state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-11-07T10:02:55Z)
• Personalization Improves Privacy-Accuracy Tradeoffs in Federated Optimization [57.98426940386627]
  We show that coordinating local learning with private centralized learning yields a generically useful and improved tradeoff between accuracy and privacy. We illustrate our theoretical results with experiments on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-02-10T20:44:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.