Personalized Federated Learning With Structure

• URL: http://arxiv.org/abs/2203.00829v3
• Date: Fri, 4 Mar 2022 07:49:04 GMT
• Title: Personalized Federated Learning With Structure
• Authors: Fengwen Chen, Guodong Longr, Zonghan Wu, Tianyi Zhou and Jing Jiang
• Abstract summary: We propose a novel structured federated learning(SFL) framework to simultaneously learn the global model and personalized model. In contrast to a pre-defined structure, our framework could be further enhanced by adding a structure learning component to automatically learn the structure.
• Score: 24.566947384179837
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Knowledge sharing and model personalization are two key components to impact the performance of personalized federated learning (PFL). Existing PFL methods simply treat knowledge sharing as an aggregation of all clients regardless of the hidden relations among them. This paper is to enhance the knowledge-sharing process in PFL by leveraging the structural information among clients. We propose a novel structured federated learning(SFL) framework to simultaneously learn the global model and personalized model using each client's local relations with others and its private dataset. This proposed framework has been formulated to a new optimization problem to model the complex relationship among personalized models and structural topology information into a unified framework. Moreover, in contrast to a pre-defined structure, our framework could be further enhanced by adding a structure learning component to automatically learn the structure using the similarities between clients' models' parameters. By conducting extensive experiments, we first demonstrate how federated learning can be benefited by introducing structural information into the server aggregation process with a real-world dataset, and then the effectiveness of the proposed method has been demonstrated in varying degrees of data non-iid settings.

Related papers

• Multi-Level Additive Modeling for Structured Non-IID Federated Learning [54.53672323071204]
  We train models organized in a multi-level structure, called Multi-level Additive Models (MAM)'', for better knowledge-sharing across heterogeneous clients. In federated MAM (FeMAM), each client is assigned to at most one model per level and its personalized prediction sums up the outputs of models assigned to it across all levels. Experiments show that FeMAM surpasses existing clustered FL and personalized FL methods in various non-IID settings.
  arXiv  Detail & Related papers  (2024-05-26T07:54:53Z)
• Decoupled Subgraph Federated Learning [57.588938805581044]
  We address the challenge of federated learning on graph-structured data distributed across multiple clients. We present a novel framework for this scenario, named FedStruct, that harnesses deep structural dependencies. We validate the effectiveness of FedStruct through experimental results conducted on six datasets for semi-supervised node classification.
  arXiv  Detail & Related papers  (2024-02-29T13:47:23Z)
• Learn What You Need in Personalized Federated Learning [53.83081622573734]
  $textitLearn2pFed$ is a novel algorithm-unrolling-based personalized federated learning framework. We show that $textitLearn2pFed$ significantly outperforms previous personalized federated learning methods.
  arXiv  Detail & Related papers  (2024-01-16T12:45:15Z)
• ARIA: On the Interaction Between Architectures, Initialization and Aggregation Methods for Federated Visual Classification [1.4585326585979703]
  We conduct the first joint ARchitecture-Initialization-Aggregation study and benchmark ARIAs across a range of medical image classification tasks. We find that, contrary to current practices, ARIA elements have to be chosen together to achieve the best possible performance. Our results shed light on good choices for each element depending on the task, the effect of normalisation layers, and the utility of SSL pre-training.
  arXiv  Detail & Related papers  (2023-11-24T17:40:31Z)
• Visual Prompt Based Personalized Federated Learning [83.04104655903846]
  We propose a novel PFL framework for image classification tasks, dubbed pFedPT, that leverages personalized visual prompts to implicitly represent local data distribution information of clients. Experiments on the CIFAR10 and CIFAR100 datasets show that pFedPT outperforms several state-of-the-art (SOTA) PFL algorithms by a large margin in various settings.
  arXiv  Detail & Related papers  (2023-03-15T15:02:15Z)
• PerFED-GAN: Personalized Federated Learning via Generative Adversarial Networks [46.17495529441229]
  Federated learning is a distributed machine learning method that can be used to deploy AI-dependent IoT applications. This paper proposes a federated learning method based on co-training and generative adversarial networks(GANs) In our experiments, the proposed method outperforms the existing methods in mean test accuracy by 42% when the client's model architecture and data distribution vary significantly.
  arXiv  Detail & Related papers  (2022-02-18T12:08:46Z)
• Architecture Agnostic Federated Learning for Neural Networks [19.813602191888837]
  This work introduces a novel Federated Heterogeneous Neural Networks (FedHeNN) framework. FedHeNN allows each client to build a personalised model without enforcing a common architecture across clients. The key idea of FedHeNN is to use the instance-level representations obtained from peer clients to guide the simultaneous training on each client.
  arXiv  Detail & Related papers  (2022-02-15T22:16:06Z)
• Parameterized Knowledge Transfer for Personalized Federated Learning [11.223753730705374]
  We propose a novel training framework to employ personalized models for different clients. It is demonstrated that the proposed framework is the first federated learning paradigm that realizes personalized model training.
  arXiv  Detail & Related papers  (2021-11-04T13:41:45Z)
• Edge-assisted Democratized Learning Towards Federated Analytics [67.44078999945722]
  We show the hierarchical learning structure of the proposed edge-assisted democratized learning mechanism, namely Edge-DemLearn. We also validate Edge-DemLearn as a flexible model training mechanism to build a distributed control and aggregation methodology in regions.
  arXiv  Detail & Related papers  (2020-12-01T11:46:03Z)
• Heterogeneous Federated Learning [41.04946606973614]
  Federated learning learns from scattered data by fusing collaborative models from local nodes. Due to chaotic information distribution, the model fusion may suffer from structural misalignment with regard to unmatched parameters. We propose a novel federated learning framework to establish a firm structure-information alignment across collaborative models.
  arXiv  Detail & Related papers  (2020-08-15T19:06:59Z)
• Federated Residual Learning [53.77128418049985]
  We study a new form of federated learning where the clients train personalized local models and make predictions jointly with the server-side shared model. Using this new federated learning framework, the complexity of the central shared model can be minimized while still gaining all the performance benefits that joint training provides.
  arXiv  Detail & Related papers  (2020-03-28T19:55:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.