FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity

• URL: http://arxiv.org/abs/2404.09816v1
• Date: Mon, 15 Apr 2024 14:14:05 GMT
• Title: FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity
• Authors: Kai Yi, Nidham Gazagnadou, Peter Richtárik, Lingjuan Lyu,
• Abstract summary: We present an effective and adaptable federated framework FedP3, representing Federated Personalized and Privacy-friendly network Pruning. We offer a theoretical interpretation of FedP3 and its locally differential-private variant, DP-FedP3, and theoretically validate their efficiencies.
• Score: 82.5448598805968
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The interest in federated learning has surged in recent research due to its unique ability to train a global model using privacy-secured information held locally on each client. This paper pays particular attention to the issue of client-side model heterogeneity, a pervasive challenge in the practical implementation of FL that escalates its complexity. Assuming a scenario where each client possesses varied memory storage, processing capabilities and network bandwidth - a phenomenon referred to as system heterogeneity - there is a pressing need to customize a unique model for each client. In response to this, we present an effective and adaptable federated framework FedP3, representing Federated Personalized and Privacy-friendly network Pruning, tailored for model heterogeneity scenarios. Our proposed methodology can incorporate and adapt well-established techniques to its specific instances. We offer a theoretical interpretation of FedP3 and its locally differential-private variant, DP-FedP3, and theoretically validate their efficiencies.

Related papers

• Personalized federated learning based on feature fusion [2.943623084019036]
  Federated learning enables distributed clients to collaborate on training while storing their data locally to protect client privacy. We propose a personalized federated learning approach called pFedPM. In our process, we replace traditional gradient uploading with feature uploading, which helps reduce communication costs and allows for heterogeneous client models.
  arXiv  Detail & Related papers  (2024-06-24T12:16:51Z)
• Towards Personalized Federated Learning via Heterogeneous Model Reassembly [84.44268421053043]
  pFedHR is a framework that leverages heterogeneous model reassembly to achieve personalized federated learning. pFedHR dynamically generates diverse personalized models in an automated manner.
  arXiv  Detail & Related papers  (2023-08-16T19:36:01Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• Performative Federated Learning: A Solution to Model-Dependent and Heterogeneous Distribution Shifts [24.196279060605402]
  We consider a federated learning (FL) system consisting of multiple clients and a server. Unlike the conventional FL framework that assumes the client's data is static, we consider scenarios where the clients' data distributions may be reshaped by the deployed decision model.
  arXiv  Detail & Related papers  (2023-05-08T23:29:24Z)
• PaDPaF: Partial Disentanglement with Partially-Federated GANs [5.195669033269619]
  Federated learning has become a popular machine learning paradigm with many potential real-life applications. This work proposes a novel architecture combining global client-agnostic and local client-specific generative models. We show that our proposed model achieves privacy and personalization by implicitly disentangling the globally consistent representation.
  arXiv  Detail & Related papers  (2022-12-07T18:28:54Z)
• Personalizing or Not: Dynamically Personalized Federated Learning with Incentives [37.42347737911428]
  We propose personalized federated learning (FL) for learning personalized models without sharing private data. We introduce the personalization rate, measured as the fraction of clients willing to train personalized models, into federated settings and propose DyPFL. This technique incentivizes clients to participate in personalizing local models while allowing the adoption of the global model when it performs better.
  arXiv  Detail & Related papers  (2022-08-12T09:51:20Z)
• Straggler-Resilient Personalized Federated Learning [55.54344312542944]
  Federated learning allows training models from samples distributed across a large network of clients while respecting privacy and communication restrictions. We develop a novel algorithmic procedure with theoretical speedup guarantees that simultaneously handles two of these hurdles. Our method relies on ideas from representation learning theory to find a global common representation using all clients' data and learn a user-specific set of parameters leading to a personalized solution for each client.
  arXiv  Detail & Related papers  (2022-06-05T01:14:46Z)
• PFA: Privacy-preserving Federated Adaptation for Effective Model Personalization [6.66389628571674]
  Federated learning (FL) has become a prevalent distributed machine learning paradigm with improved privacy. This paper introduces a new concept called federated adaptation, targeting at adapting the trained model in a federated manner to achieve better personalization results. We propose PFA, a framework to accomplish Privacy-preserving Federated Adaptation.
  arXiv  Detail & Related papers  (2021-03-02T08:07:34Z)
• Toward Understanding the Influence of Individual Clients in Federated Learning [52.07734799278535]
  Federated learning allows clients to jointly train a global model without sending their private data to a central server. We defined a new notion called em-Influence, quantify this influence over parameters, and proposed an effective efficient model to estimate this metric.
  arXiv  Detail & Related papers  (2020-12-20T14:34:36Z)
• Federated Mutual Learning [65.46254760557073]
  Federated Mutual Leaning (FML) allows clients training a generalized model collaboratively and a personalized model independently. The experiments show that FML can achieve better performance than alternatives in typical Federated learning setting.
  arXiv  Detail & Related papers  (2020-06-27T09:35:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.