Hypernetworks for Model-Heterogeneous Personalized Federated Learning

• URL: http://arxiv.org/abs/2507.22330v1
• Date: Wed, 30 Jul 2025 02:24:26 GMT
• Title: Hypernetworks for Model-Heterogeneous Personalized Federated Learning
• Authors: Chen Zhang, Husheng Li, Xiang Liu, Linshan Jiang, Danxin Wang,
• Abstract summary: We propose a server-side hypernetwork that takes client-specific embedding vectors as input and outputs personalized parameters tailored to each client's heterogeneous model.<n>To promote knowledge sharing and reduce computation, we introduce a multi-head structure within the hypernetwork, allowing clients with similar model sizes to share heads.<n>Our framework does not rely on external datasets and does not require disclosure of client model architectures.
• Score: 13.408669475480824
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in personalized federated learning have focused on addressing client model heterogeneity. However, most existing methods still require external data, rely on model decoupling, or adopt partial learning strategies, which can limit their practicality and scalability. In this paper, we revisit hypernetwork-based methods and leverage their strong generalization capabilities to design a simple yet effective framework for heterogeneous personalized federated learning. Specifically, we propose MH-pFedHN, which leverages a server-side hypernetwork that takes client-specific embedding vectors as input and outputs personalized parameters tailored to each client's heterogeneous model. To promote knowledge sharing and reduce computation, we introduce a multi-head structure within the hypernetwork, allowing clients with similar model sizes to share heads. Furthermore, we further propose MH-pFedHNGD, which integrates an optional lightweight global model to improve generalization. Our framework does not rely on external datasets and does not require disclosure of client model architectures, thereby offering enhanced privacy and flexibility. Extensive experiments on multiple benchmarks and model settings demonstrate that our approach achieves competitive accuracy, strong generalization, and serves as a robust baseline for future research in model-heterogeneous personalized federated learning.

Related papers

• Not All Clients Are Equal: Personalized Federated Learning on Heterogeneous Multi-Modal Clients [52.14230635007546]
  Foundation models have shown remarkable capabilities across diverse multi-modal tasks, but their centralized training raises privacy concerns and induces high transmission costs.<n>For the growing demand for personalizing AI models for different user purposes, personalized federated learning (PFL) has emerged.<n>PFL allows each client to leverage the knowledge of other clients for further adaptation to individual user preferences, again without the need to share data.
  arXiv  Detail & Related papers  (2025-05-20T09:17:07Z)
• FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors [50.131271229165165]
  Federated Learning (FL) has emerged as a promising framework for distributed machine learning.<n>Data heterogeneity resulting from differences across user behaviors, preferences, and device characteristics poses a significant challenge for federated learning.<n>We propose Adaptive Weight Aggregation (FedAWA), a novel method that adaptively adjusts aggregation weights based on client vectors during the learning process.
  arXiv  Detail & Related papers  (2025-03-20T04:49:40Z)
• Client-Centric Federated Adaptive Optimization [78.30827455292827]
  Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private.<n>We propose Federated-Centric Adaptive Optimization, which is a class of novel federated optimization approaches.
  arXiv  Detail & Related papers  (2025-01-17T04:00:50Z)
• GeFL: Model-Agnostic Federated Learning with Generative Models [3.4546761246181696]
  Federated learning (FL) is a distributed training paradigm that enables collaborative learning across clients without sharing local data, thereby preserving privacy.<n>We propose Generative Model-Aided Federated Learning (GeFL), a framework that enables cross-client knowledge sharing via a generative model trained in a federated manner.
  arXiv  Detail & Related papers  (2024-12-24T14:39:47Z)
• Personalized Hierarchical Split Federated Learning in Wireless Networks [24.664469755746463]
  We propose a personalized hierarchical split federated learning (PHSFL) algorithm that is specially designed to achieve better personalization performance.<n>We first perform extensive theoretical analysis to understand the impact of model splitting and hierarchical model aggregations on the global model.<n>Once the global model is trained, we fine-tune each client to obtain the personalized models.
  arXiv  Detail & Related papers  (2024-11-09T02:41:53Z)
• Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boosting [76.64235084279292]
  One-shot Federated Learning (OFL) has become a promising learning paradigm, enabling the training of a global server model via a single communication round. We introduce a novel framework, Co-Boosting, in which synthesized data and the ensemble model mutually enhance each other progressively.
  arXiv  Detail & Related papers  (2024-02-23T03:15:10Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Personalizing Federated Learning with Over-the-Air Computations [84.8089761800994]
  Federated edge learning is a promising technology to deploy intelligence at the edge of wireless networks in a privacy-preserving manner. Under such a setting, multiple clients collaboratively train a global generic model under the coordination of an edge server. This paper presents a distributed training paradigm that employs analog over-the-air computation to address the communication bottleneck.
  arXiv  Detail & Related papers  (2023-02-24T08:41:19Z)
• PaDPaF: Partial Disentanglement with Partially-Federated GANs [5.195669033269619]
  Federated learning has become a popular machine learning paradigm with many potential real-life applications. This work proposes a novel architecture combining global client-agnostic and local client-specific generative models. We show that our proposed model achieves privacy and personalization by implicitly disentangling the globally consistent representation.
  arXiv  Detail & Related papers  (2022-12-07T18:28:54Z)
• Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learning [22.310090483499035]
  Federated learning (FL) enables edge-devices to collaboratively learn a model without disclosing their private data to a central aggregating server. Most existing FL algorithms require models of identical architecture to be deployed across the clients and server. We propose a novel ensemble knowledge transfer method named Fed-ET in which small models are trained on clients, and used to train a larger model at the server.
  arXiv  Detail & Related papers  (2022-04-27T05:18:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.