FedSheafHN: Personalized Federated Learning on Graph-structured Data

• URL: http://arxiv.org/abs/2405.16056v3
• Date: Fri, 31 May 2024 11:44:39 GMT
• Title: FedSheafHN: Personalized Federated Learning on Graph-structured Data
• Authors: Wenfei Liang, Yanan Zhao, Rui She, Yiming Li, Wee Peng Tay,
• Abstract summary: We propose a model called FedSheafHN, which embeds each client's local subgraph into a server-constructed collaboration graph. Our model improves the integration and interpretation of complex client characteristics. It also has fast model convergence and effective new clients generalization.
• Score: 22.825083541211168
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Personalized subgraph Federated Learning (FL) is a task that customizes Graph Neural Networks (GNNs) to individual client needs, accommodating diverse data distributions. However, applying hypernetworks in FL, while aiming to facilitate model personalization, often encounters challenges due to inadequate representation of client-specific characteristics. To overcome these limitations, we propose a model called FedSheafHN, using enhanced collaboration graph embedding and efficient personalized model parameter generation. Specifically, our model embeds each client's local subgraph into a server-constructed collaboration graph. We utilize sheaf diffusion in the collaboration graph to learn client representations. Our model improves the integration and interpretation of complex client characteristics. Furthermore, our model ensures the generation of personalized models through advanced hypernetworks optimized for parallel operations across clients. Empirical evaluations demonstrate that FedSheafHN outperforms existing methods in most scenarios, in terms of client model performance on various graph-structured datasets. It also has fast model convergence and effective new clients generalization.

Related papers

• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• Rethinking Personalized Federated Learning with Clustering-based Dynamic Graph Propagation [48.08348593449897]
  We propose a simple yet effective personalized federated learning framework. We group clients into multiple clusters based on their model training status and data distribution on the server side. We conduct experiments on three image benchmark datasets and create synthetic structured datasets with three types of typologies.
  arXiv  Detail & Related papers  (2024-01-29T04:14:02Z)
• Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generation [38.42808389088285]
  Federated learning (FL) emerges as a decentralized learning framework which trains models from multiple distributed clients without sharing their data to preserve privacy. We propose a novel personalized FL framework of client-specific Prompt Generation (pFedPG) pFedPG learns to deploy a personalized prompt generator at the server for producing client-specific visual prompts that efficiently adapts frozen backbones to local data distributions.
  arXiv  Detail & Related papers  (2023-08-29T15:03:05Z)
• FAM: fast adaptive federated meta-learning [10.980548731600116]
  We propose a fast adaptive federated meta-learning (FAM) framework for collaboratively learning a single global model. A skeleton network is grown on each client to train a personalized model by learning additional client-specific parameters from local data. The personalized client models outperformed the locally trained models, demonstrating the efficacy of the FAM mechanism.
  arXiv  Detail & Related papers  (2023-08-26T22:54:45Z)
• Visual Prompt Based Personalized Federated Learning [83.04104655903846]
  We propose a novel PFL framework for image classification tasks, dubbed pFedPT, that leverages personalized visual prompts to implicitly represent local data distribution information of clients. Experiments on the CIFAR10 and CIFAR100 datasets show that pFedPT outperforms several state-of-the-art (SOTA) PFL algorithms by a large margin in various settings.
  arXiv  Detail & Related papers  (2023-03-15T15:02:15Z)
• Personalizing Federated Learning with Over-the-Air Computations [84.8089761800994]
  Federated edge learning is a promising technology to deploy intelligence at the edge of wireless networks in a privacy-preserving manner. Under such a setting, multiple clients collaboratively train a global generic model under the coordination of an edge server. This paper presents a distributed training paradigm that employs analog over-the-air computation to address the communication bottleneck.
  arXiv  Detail & Related papers  (2023-02-24T08:41:19Z)
• QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning [8.420943739336067]
  federated learning (FL) aims to train a single global model while collaboratively using multiple clients and a server. We introduce a textitquantized and textitpersonalized FL algorithm QuPeD that facilitates collective (personalized model compression) training. Numerically, we validate that QuPeD outperforms competing personalized FL methods, FedAvg, and local training of clients in various heterogeneous settings.
  arXiv  Detail & Related papers  (2021-07-29T10:55:45Z)
• Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneity [3.291862617649511]
  We propose a new approach for obtaining a personalized model from a client-level objective. To realize this personalization, we leverage finding a small subnetwork for each client.
  arXiv  Detail & Related papers  (2021-05-02T22:10:46Z)
• Toward Understanding the Influence of Individual Clients in Federated Learning [52.07734799278535]
  Federated learning allows clients to jointly train a global model without sending their private data to a central server. We defined a new notion called em-Influence, quantify this influence over parameters, and proposed an effective efficient model to estimate this metric.
  arXiv  Detail & Related papers  (2020-12-20T14:34:36Z)
• Personalized Federated Learning with First Order Model Optimization [76.81546598985159]
  We propose an alternative to federated learning, where each client federates with other relevant clients to obtain a stronger model per client-specific objectives. We do not assume knowledge of underlying data distributions or client similarities, and allow each client to optimize for arbitrary target distributions of interest. Our method outperforms existing alternatives, while also enabling new features for personalized FL such as transfer outside of local data distributions.
  arXiv  Detail & Related papers  (2020-12-15T19:30:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.