FedAGHN: Personalized Federated Learning with Attentive Graph HyperNetworks

• URL: http://arxiv.org/abs/2501.16379v1
• Date: Fri, 24 Jan 2025 10:48:30 GMT
• Title: FedAGHN: Personalized Federated Learning with Attentive Graph HyperNetworks
• Authors: Jiarui Song, Yunheng Shen, Chengbin Hou, Pengyu Wang, Jinbao Wang, Ke Tang, Hairong Lv,
• Abstract summary: PFL aims to address the statistical heterogeneity of data across clients by learning the personalized model for each client. We propose Personalized Federated Learning with Attentive Graph HyperNetworks (FedAGHN) FedAGHN captures fine-grained collaborative relationships and generates client-specific personalized initial models.
• Score: 19.57993976799076
• License:
• Abstract: Personalized Federated Learning (PFL) aims to address the statistical heterogeneity of data across clients by learning the personalized model for each client. Among various PFL approaches, the personalized aggregation-based approach conducts parameter aggregation in the server-side aggregation phase to generate personalized models, and focuses on learning appropriate collaborative relationships among clients for aggregation. However, the collaborative relationships vary in different scenarios and even at different stages of the FL process. To this end, we propose Personalized Federated Learning with Attentive Graph HyperNetworks (FedAGHN), which employs Attentive Graph HyperNetworks (AGHNs) to dynamically capture fine-grained collaborative relationships and generate client-specific personalized initial models. Specifically, AGHNs empower graphs to explicitly model the client-specific collaborative relationships, construct collaboration graphs, and introduce tunable attentive mechanism to derive the collaboration weights, so that the personalized initial models can be obtained by aggregating parameters over the collaboration graphs. Extensive experiments can demonstrate the superiority of FedAGHN. Moreover, a series of visualizations are presented to explore the effectiveness of collaboration graphs learned by FedAGHN.

Related papers

• Toward Model-centric Heterogeneous Federated Graph Learning: A Knowledge-driven Approach [15.747994926672868]
  Federated graph learning (FGL) has emerged as a promising paradigm for collaborative machine learning. MHtFGL problem arises in real-world applications, such as the aggregation of models from different companies. We propose the Federated Graph Knowledge Collaboration (FedGKC) framework, comprising two key components.
  arXiv  Detail & Related papers  (2025-01-22T04:12:32Z)
• Personalized Federated Knowledge Graph Embedding with Client-Wise Relation Graph [49.66272783945571]
  We propose Personalized Federated knowledge graph Embedding with client-wise relation graph. PFedEG learns personalized supplementary knowledge for each client by amalgamating entity embedding from its neighboring clients. We conduct extensive experiments on four benchmark datasets to evaluate our method against state-of-the-art models.
  arXiv  Detail & Related papers  (2024-06-17T17:44:53Z)
• FedSheafHN: Personalized Federated Learning on Graph-structured Data [22.825083541211168]
  We propose a model called FedSheafHN, which embeds each client's local subgraph into a server-constructed collaboration graph. Our model improves the integration and interpretation of complex client characteristics. It also has fast model convergence and effective new clients generalization.
  arXiv  Detail & Related papers  (2024-05-25T04:51:41Z)
• Balancing Similarity and Complementarity for Federated Learning [91.65503655796603]
  Federated Learning (FL) is increasingly important in mobile and IoT systems. One key challenge in FL is managing statistical heterogeneity, such as non-i.i.d. data. We introduce a novel framework, textttFedSaC, which balances similarity and complementarity in FL cooperation.
  arXiv  Detail & Related papers  (2024-05-16T08:16:19Z)
• Rethinking Personalized Federated Learning with Clustering-based Dynamic Graph Propagation [48.08348593449897]
  We propose a simple yet effective personalized federated learning framework. We group clients into multiple clusters based on their model training status and data distribution on the server side. We conduct experiments on three image benchmark datasets and create synthetic structured datasets with three types of typologies.
  arXiv  Detail & Related papers  (2024-01-29T04:14:02Z)
• APGL4SR: A Generic Framework with Adaptive and Personalized Global Collaborative Information in Sequential Recommendation [86.29366168836141]
  We propose a graph-driven framework, named Adaptive and Personalized Graph Learning for Sequential Recommendation (APGL4SR) APGL4SR incorporates adaptive and personalized global collaborative information into sequential recommendation systems. As a generic framework, APGL4SR can outperform other baselines with significant margins.
  arXiv  Detail & Related papers  (2023-11-06T01:33:24Z)
• Structured Cooperative Learning with Graphical Model Priors [98.53322192624594]
  We study how to train personalized models for different tasks on decentralized devices with limited local data. We propose "Structured Cooperative Learning (SCooL)", in which a cooperation graph across devices is generated by a graphical model. We evaluate SCooL and compare it with existing decentralized learning methods on an extensive set of benchmarks.
  arXiv  Detail & Related papers  (2023-06-16T02:41:31Z)
• Unrolled Graph Learning for Multi-Agent Collaboration [37.239120967721156]
  We propose a distributed multi-agent learning model inspired by human collaboration. Agents can autonomously detect suitable collaborators and refer to collaborators' model for better performance.
  arXiv  Detail & Related papers  (2022-10-31T07:05:44Z)
• A Coalition Formation Game Approach for Personalized Federated Learning [12.784305390534888]
  We propose a novel personalized algorithm: pFedSV, which can 1. identify each client's optimal collaborator coalition and 2. perform personalized model aggregation based on SV. The results show that pFedSV can achieve superior personalized accuracy for each client, compared to the state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2022-02-05T07:16:44Z)
• Tensor Graph Convolutional Networks for Multi-relational and Robust Learning [74.05478502080658]
  This paper introduces a tensor-graph convolutional network (TGCN) for scalable semi-supervised learning (SSL) from data associated with a collection of graphs, that are represented by a tensor. The proposed architecture achieves markedly improved performance relative to standard GCNs, copes with state-of-the-art adversarial attacks, and leads to remarkable SSL performance over protein-to-protein interaction networks.
  arXiv  Detail & Related papers  (2020-03-15T02:33:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.