Privacy and Accuracy Implications of Model Complexity and Integration in Heterogeneous Federated Learning

• URL: http://arxiv.org/abs/2311.17750v3
• Date: Mon, 10 Mar 2025 11:10:50 GMT
• Title: Privacy and Accuracy Implications of Model Complexity and Integration in Heterogeneous Federated Learning
• Authors: Gergely Dániel Németh, Miguel Ángel Lozano, Novi Quadrianto, Nuria Oliver,
• Abstract summary: Federated Learning (FL) has been proposed as a privacy-preserving solution for distributed machine learning.<n>Recent studies have shown that it is susceptible to membership inference attacks (MIA), which can compromise the privacy of client data.
• Score: 8.842172558292027
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning (FL) has been proposed as a privacy-preserving solution for distributed machine learning, particularly in heterogeneous FL settings where clients have varying computational capabilities and thus train models with different complexities compared to the server's model. However, FL is not without vulnerabilities: recent studies have shown that it is susceptible to membership inference attacks (MIA), which can compromise the privacy of client data. In this paper, we examine the intersection of these two aspects, heterogeneous FL and its privacy vulnerabilities, by focusing on the role of client model integration, the process through which the server integrates parameters from clients' smaller models into its larger model. To better understand this process, we first propose a taxonomy that categorizes existing heterogeneous FL methods and enables the design of seven novel heterogeneous FL model integration strategies. Using CIFAR-10, CIFAR-100, and FEMNIST vision datasets, we evaluate the privacy and accuracy trade-offs of these approaches under three types of MIAs. Our findings reveal significant differences in privacy leakage and performance depending on the integration method. Notably, introducing randomness in the model integration process enhances client privacy while maintaining competitive accuracy for both the clients and the server. This work provides quantitative light on the privacy-accuracy implications client model integration in heterogeneous FL settings, paving the way towards more secure and efficient FL systems.

Related papers

• Secure Generalization through Stochastic Bidirectional Parameter Updates Using Dual-Gradient Mechanism [6.03163048890944]
  Federated learning (FL) has gained increasing attention due to privacy-preserving collaborative training on decentralized clients. Recent research has underscored the risk of exposing private data to adversaries, even within FL frameworks. We generate diverse models for each client by using systematic perturbations in model parameters at a fine-grained level.
  arXiv  Detail & Related papers  (2025-04-03T02:06:57Z)
• FedConv: A Learning-on-Model Paradigm for Heterogeneous Federated Clients [25.847042398060616]
  Federated Learning (FL) facilitates collaborative training of a shared global model without exposing clients' private data. We propose FedConv, a client-friendly FL framework, which minimizes the computation and memory burden on resource-constrained clients. We show that FedConv outperforms state-of-the-art FL systems in terms of model accuracy, computation and communication overhead.
  arXiv  Detail & Related papers  (2025-02-28T01:39:53Z)
• Client-Centric Federated Adaptive Optimization [78.30827455292827]
  Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private. We propose Federated-Centric Adaptive Optimization, which is a class of novel federated optimization approaches.
  arXiv  Detail & Related papers  (2025-01-17T04:00:50Z)
• Robust Federated Learning in the Face of Covariate Shift: A Magnitude Pruning with Hybrid Regularization Framework for Enhanced Model Aggregation [1.519321208145928]
  Federated Learning (FL) offers a promising framework for individuals aiming to collaboratively develop a shared model. variations in data distribution among clients can profoundly affect FL methodologies, primarily due to instabilities in the aggregation process. We propose a novel FL framework, combining individual parameter pruning and regularization techniques to improve the robustness of individual clients' models to aggregate.
  arXiv  Detail & Related papers  (2024-12-19T16:22:37Z)
• Immersion and Invariance-based Coding for Privacy-Preserving Federated Learning [1.4226399196408985]
  Federated learning (FL) has emerged as a method to preserve privacy in collaborative distributed learning. We introduce a privacy-preserving FL framework that combines differential privacy and system immersion tools from control theory. We demonstrate that the proposed privacy-preserving scheme can be tailored to offer any desired level of differential privacy for both local and global model parameters.
  arXiv  Detail & Related papers  (2024-09-25T15:04:42Z)
• Federated Behavioural Planes: Explaining the Evolution of Client Behaviour in Federated Learning [6.64590374742412]
  We introduce Federated Behavioural Planes (FBPs), a novel method to analyse, visualise, and explain the dynamics of FL systems. Our experiments demonstrate that FBPs provide informative trajectories describing the evolving states of clients. We propose a robust aggregation technique named Federated Behavioural Shields to detect malicious or noisy client models.
  arXiv  Detail & Related papers  (2024-05-24T15:17:51Z)
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• FLASH: Federated Learning Across Simultaneous Heterogeneities [54.80435317208111]
  FLASH(Federated Learning Across Simultaneous Heterogeneities) is a lightweight and flexible client selection algorithm. It outperforms state-of-the-art FL frameworks under extensive sources of Heterogeneities. It achieves substantial and consistent improvements over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-13T20:04:39Z)
• Contrastive encoder pre-training-based clustered federated learning for heterogeneous data [17.580390632874046]
  Federated learning (FL) enables distributed clients to collaboratively train a global model while preserving their data privacy. We propose contrastive pre-training-based clustered federated learning (CP-CFL) to improve the model convergence and overall performance of FL systems.
  arXiv  Detail & Related papers  (2023-11-28T05:44:26Z)
• PFL-GAN: When Client Heterogeneity Meets Generative Models in Personalized Federated Learning [55.930403371398114]
  We propose a novel generative adversarial network (GAN) sharing and aggregation strategy for personalized learning (PFL) PFL-GAN addresses the client heterogeneity in different scenarios. More specially, we first learn the similarity among clients and then develop an weighted collaborative data aggregation. The empirical results through the rigorous experimentation on several well-known datasets demonstrate the effectiveness of PFL-GAN.
  arXiv  Detail & Related papers  (2023-08-23T22:38:35Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• Blockchain-based Optimized Client Selection and Privacy Preserved Framework for Federated Learning [2.4201849657206496]
  Federated learning is a distributed mechanism that trained large-scale neural network models with the participation of multiple clients. With this feature, federated learning is considered a secure solution for data privacy issues. We proposed the blockchain-based optimized client selection and privacy-preserved framework.
  arXiv  Detail & Related papers  (2023-07-25T01:35:51Z)
• Confidence-aware Personalized Federated Learning via Variational Expectation Maximization [34.354154518009956]
  We present a novel framework for personalized Federated Learning (PFL) PFL is a distributed learning scheme to train a shared model across clients. We present a novel framework for PFL based on hierarchical modeling and variational inference.
  arXiv  Detail & Related papers  (2023-05-21T20:12:27Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• Visual Prompt Based Personalized Federated Learning [83.04104655903846]
  We propose a novel PFL framework for image classification tasks, dubbed pFedPT, that leverages personalized visual prompts to implicitly represent local data distribution information of clients. Experiments on the CIFAR10 and CIFAR100 datasets show that pFedPT outperforms several state-of-the-art (SOTA) PFL algorithms by a large margin in various settings.
  arXiv  Detail & Related papers  (2023-03-15T15:02:15Z)
• Closing the Gap between Client and Global Model Performance in Heterogeneous Federated Learning [2.1044900734651626]
  We show how the chosen approach for training custom client models has an impact on the global model. We propose a new approach that combines KD and Learning without Forgetting (LwoF) to produce improved personalised models.
  arXiv  Detail & Related papers  (2022-11-07T11:12:57Z)
• Toward Understanding the Influence of Individual Clients in Federated Learning [52.07734799278535]
  Federated learning allows clients to jointly train a global model without sending their private data to a central server. We defined a new notion called em-Influence, quantify this influence over parameters, and proposed an effective efficient model to estimate this metric.
  arXiv  Detail & Related papers  (2020-12-20T14:34:36Z)
• Federated Mutual Learning [65.46254760557073]
  Federated Mutual Leaning (FML) allows clients training a generalized model collaboratively and a personalized model independently. The experiments show that FML can achieve better performance than alternatives in typical Federated learning setting.
  arXiv  Detail & Related papers  (2020-06-27T09:35:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.