FedADP: Unified Model Aggregation for Federated Learning with Heterogeneous Model Architectures

• URL: http://arxiv.org/abs/2505.06497v1
• Date: Sat, 10 May 2025 02:57:07 GMT
• Title: FedADP: Unified Model Aggregation for Federated Learning with Heterogeneous Model Architectures
• Authors: Jiacheng Wang, Hongtao Lv, Lei Liu,
• Abstract summary: Traditional Federated Learning (FL) faces significant challenges in terms of efficiency and accuracy, particularly in heterogeneous environments.<n>We propose FedADP, a federated learning framework designed to adapt to client heterogeneity by dynamically adjusting model architectures during aggregation.<n>Our experimental results demonstrate that FedADP significantly outperforms existing methods, such as FlexiFed, achieving an accuracy improvement of up to 23.30%.
• Score: 5.348839333572149
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional Federated Learning (FL) faces significant challenges in terms of efficiency and accuracy, particularly in heterogeneous environments where clients employ diverse model architectures and have varying computational resources. Such heterogeneity complicates the aggregation process, leading to performance bottlenecks and reduced model generalizability. To address these issues, we propose FedADP, a federated learning framework designed to adapt to client heterogeneity by dynamically adjusting model architectures during aggregation. FedADP enables effective collaboration among clients with differing capabilities, maximizing resource utilization and ensuring model quality. Our experimental results demonstrate that FedADP significantly outperforms existing methods, such as FlexiFed, achieving an accuracy improvement of up to 23.30%, thereby enhancing model adaptability and training efficiency in heterogeneous real-world settings.

Related papers

• Not All Clients Are Equal: Personalized Federated Learning on Heterogeneous Multi-Modal Clients [52.14230635007546]
  Foundation models have shown remarkable capabilities across diverse multi-modal tasks, but their centralized training raises privacy concerns and induces high transmission costs.<n>For the growing demand for personalizing AI models for different user purposes, personalized federated learning (PFL) has emerged.<n>PFL allows each client to leverage the knowledge of other clients for further adaptation to individual user preferences, again without the need to share data.
  arXiv  Detail & Related papers  (2025-05-20T09:17:07Z)
• Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clients [60.22876915395139]
  This paper studies a challenging robust federated learning task with model heterogeneous and data corrupted clients.<n>Data corruption is unavoidable due to factors such as random noise, compression artifacts, or environmental conditions in real-world deployment.<n>We propose a novel Robust Asymmetric Heterogeneous Federated Learning framework to address these issues.
  arXiv  Detail & Related papers  (2025-03-12T09:52:04Z)
• Client-Centric Federated Adaptive Optimization [78.30827455292827]
  Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private.<n>We propose Federated-Centric Adaptive Optimization, which is a class of novel federated optimization approaches.
  arXiv  Detail & Related papers  (2025-01-17T04:00:50Z)
• FedDUAL: A Dual-Strategy with Adaptive Loss and Dynamic Aggregation for Mitigating Data Heterogeneity in Federated Learning [12.307490659840845]
  Federated Learning (FL) combines locally optimized models from various clients into a unified global model.<n>FL encounters significant challenges such as performance degradation, slower convergence, and reduced robustness of the global model.<n>We introduce an innovative dual-strategy approach designed to effectively resolve these issues.
  arXiv  Detail & Related papers  (2024-12-05T18:42:29Z)
• Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architectures [15.645254436094055]
  Federated Learning (FL) enables collaborative fine-tuning of Large Language Models without data sharing.<n>We propose FedAMoLE, a lightweight personalized FL framework that enables data-driven heterogeneous model architectures.<n> Experiments show that FedAMoLE improves accuracy by an average of 5.14% compared to existing approaches.
  arXiv  Detail & Related papers  (2024-11-28T13:20:38Z)
• Towards Efficient Model-Heterogeneity Federated Learning for Large Models [18.008063521900702]
  We introduce HeteroTune, an innovative fine-tuning framework tailored for model-heterogeneity federated learning (MHFL) In particular, we propose a novel parameter-efficient fine-tuning structure, called FedAdapter, which employs a multi-branch cross-model aggregator. Benefiting from the lightweight FedAdapter, our approach significantly reduces both the computational and communication overhead.
  arXiv  Detail & Related papers  (2024-11-25T09:58:51Z)
• FedPAE: Peer-Adaptive Ensemble Learning for Asynchronous and Model-Heterogeneous Federated Learning [9.084674176224109]
  Federated learning (FL) enables multiple clients with distributed data sources to collaboratively train a shared model without compromising data privacy. We introduce Federated Peer-Adaptive Ensemble Learning (FedPAE), a fully decentralized pFL algorithm that supports model heterogeneity and asynchronous learning. Our approach utilizes a peer-to-peer model sharing mechanism and ensemble selection to achieve a more refined balance between local and global information.
  arXiv  Detail & Related papers  (2024-10-17T22:47:19Z)
• Towards Personalized Federated Learning via Heterogeneous Model Reassembly [84.44268421053043]
  pFedHR is a framework that leverages heterogeneous model reassembly to achieve personalized federated learning. pFedHR dynamically generates diverse personalized models in an automated manner.
  arXiv  Detail & Related papers  (2023-08-16T19:36:01Z)
• Slimmable Domain Adaptation [112.19652651687402]
  We introduce a simple framework, Slimmable Domain Adaptation, to improve cross-domain generalization with a weight-sharing model bank. Our framework surpasses other competing approaches by a very large margin on multiple benchmarks.
  arXiv  Detail & Related papers  (2022-06-14T06:28:04Z)
• Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learning [22.310090483499035]
  Federated learning (FL) enables edge-devices to collaboratively learn a model without disclosing their private data to a central aggregating server. Most existing FL algorithms require models of identical architecture to be deployed across the clients and server. We propose a novel ensemble knowledge transfer method named Fed-ET in which small models are trained on clients, and used to train a larger model at the server.
  arXiv  Detail & Related papers  (2022-04-27T05:18:32Z)
• Dynamic Federated Learning [57.14673504239551]
  Federated learning has emerged as an umbrella term for centralized coordination strategies in multi-agent environments. We consider a federated learning model where at every iteration, a random subset of available agents perform local updates based on their data. Under a non-stationary random walk model on the true minimizer for the aggregate optimization problem, we establish that the performance of the architecture is determined by three factors, namely, the data variability at each agent, the model variability across all agents, and a tracking term that is inversely proportional to the learning rate of the algorithm.
  arXiv  Detail & Related papers  (2020-02-20T15:00:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.