The Power of Bias: Optimizing Client Selection in Federated Learning with Heterogeneous Differential Privacy

• URL: http://arxiv.org/abs/2408.08642v1
• Date: Fri, 16 Aug 2024 10:19:27 GMT
• Title: The Power of Bias: Optimizing Client Selection in Federated Learning with Heterogeneous Differential Privacy
• Authors: Jiating Ma, Yipeng Zhou, Qi Li, Quan Z. Sheng, Laizhong Cui, Jiangchuan Liu,
• Abstract summary: Both data quality and influence of DP noises should be taken into account when selecting clients. An experiment results with real datasets under both convex and non- convex loss functions.
• Score: 38.55420329607416
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To preserve the data privacy, the federated learning (FL) paradigm emerges in which clients only expose model gradients rather than original data for conducting model training. To enhance the protection of model gradients in FL, differentially private federated learning (DPFL) is proposed which incorporates differentially private (DP) noises to obfuscate gradients before they are exposed. Yet, an essential but largely overlooked problem in DPFL is the heterogeneity of clients' privacy requirement, which can vary significantly between clients and extremely complicates the client selection problem in DPFL. In other words, both the data quality and the influence of DP noises should be taken into account when selecting clients. To address this problem, we conduct convergence analysis of DPFL under heterogeneous privacy, a generic client selection strategy, popular DP mechanisms and convex loss. Based on convergence analysis, we formulate the client selection problem to minimize the value of loss function in DPFL with heterogeneous privacy, which is a convex optimization problem and can be solved efficiently. Accordingly, we propose the DPFL-BCS (biased client selection) algorithm. The extensive experiment results with real datasets under both convex and non-convex loss functions indicate that DPFL-BCS can remarkably improve model utility compared with the SOTA baselines.

Related papers

• Client-Centric Federated Adaptive Optimization [78.30827455292827]
  Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private. We propose Federated-Centric Adaptive Optimization, which is a class of novel federated optimization approaches.
  arXiv  Detail & Related papers  (2025-01-17T04:00:50Z)
• Upcycling Noise for Federated Unlearning [10.943200894066125]
  Federated Unlearning with Indistinguishability (FUI) FuI consists of two main steps: local model retraction and global noise calibration. FuI achieves superior model performance and higher efficiency compared to mainstream FU schemes.
  arXiv  Detail & Related papers  (2024-12-07T04:07:40Z)
• Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning [21.27813247914949]
  We propose Robust-HDP, which efficiently estimates the true noise level in clients model updates. It improves utility and convergence speed, while being safe to the clients that may maliciously send falsified privacy parameter to server.
  arXiv  Detail & Related papers  (2024-06-05T17:41:42Z)
• Differentially Private Clustered Federated Learning [4.768272342753616]
  Federated learning (FL) often incorporates differential privacy (DP) to provide rigorous data privacy guarantees. Previous works attempted to address high structured data heterogeneity in vanilla FL settings through clustering clients (a.k.a clustered FL) We propose an algorithm for differentially private clustered FL, which is robust to the DP noise in the system and identifies the underlying clients' clusters correctly.
  arXiv  Detail & Related papers  (2024-05-29T17:03:31Z)
• Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approach [42.59649764999974]
  Federated learning aims to infer a shared model from private and decentralized data stored locally by multiple clients. We propose a PFL algorithm named PAC-PFL for learning probabilistic models within a PAC-Bayesian framework. Our algorithm collaboratively learns a shared hyper-posterior and regards each client's posterior inference as the step personalization.
  arXiv  Detail & Related papers  (2024-01-16T13:30:37Z)
• Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification [51.04894019092156]
  Federated learning (FL) has been recognized as a rapidly growing area, where the model is trained over clients under the FL orchestration (PS) In this paper, we propose a novel primal sparification algorithm for and guarantee non-smooth FL problems. Its unique insightful properties and its analyses are also presented.
  arXiv  Detail & Related papers  (2023-10-30T14:15:47Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• Balancing Privacy Protection and Interpretability in Federated Learning [8.759803233734624]
  Federated learning (FL) aims to collaboratively train the global model in a distributed manner by sharing the model parameters from local clients to a central server. Recent studies have illustrated that FL still suffers from information leakage as adversaries try to recover the training data by analyzing shared parameters from local clients. We propose a simple yet effective adaptive differential privacy (ADP) mechanism that selectively adds noisy perturbations to the gradients of client models in FL.
  arXiv  Detail & Related papers  (2023-02-16T02:58:22Z)
• FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximations [53.268801169075836]
  We propose FedLAP-DP, a novel privacy-preserving approach for federated learning. A formal privacy analysis demonstrates that FedLAP-DP incurs the same privacy costs as typical gradient-sharing schemes. Our approach presents a faster convergence speed compared to typical gradient-sharing methods.
  arXiv  Detail & Related papers  (2023-02-02T12:56:46Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.