Communication-Efficient and Interoperable Distributed Learning

• URL: http://arxiv.org/abs/2509.22823v1
• Date: Fri, 26 Sep 2025 18:30:36 GMT
• Title: Communication-Efficient and Interoperable Distributed Learning
• Abstract summary: Collaborative learning across heterogeneous model architectures presents significant challenges in ensuring interoperability and preserving privacy.<n>We propose a communication-efficient distributed learning framework that supports model heterogeneity and enables modular composition during inference.
• Score: 27.136816067758215
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Collaborative learning across heterogeneous model architectures presents significant challenges in ensuring interoperability and preserving privacy. We propose a communication-efficient distributed learning framework that supports model heterogeneity and enables modular composition during inference. To facilitate interoperability, all clients adopt a common fusion-layer output dimension, which permits each model to be partitioned into a personalized base block and a generalized modular block. Clients share their fusion-layer outputs, keeping model parameters and architectures private. Experimental results demonstrate that the framework achieves superior communication efficiency compared to federated learning (FL) and federated split learning (FSL) baselines, while ensuring stable training performance across heterogeneous architectures.

Related papers

• Communication-Efficient and Robust Multi-Modal Federated Learning via Latent-Space Consensus [26.0108296956723]
  Federated learning (FL) enables collaborative model training across distributed devices without sharing raw data.<n>We introduce CoMFed, a Communication-Efficient Multi-Modal Federated Learning framework.<n>A latent-space regularizer aligns these representations across clients, improving cross-modal consistency and robustness to outliers.
  arXiv  Detail & Related papers  (2026-03-19T15:56:28Z)
• FedADP: Unified Model Aggregation for Federated Learning with Heterogeneous Model Architectures [5.348839333572149]
  Traditional Federated Learning (FL) faces significant challenges in terms of efficiency and accuracy, particularly in heterogeneous environments.<n>We propose FedADP, a federated learning framework designed to adapt to client heterogeneity by dynamically adjusting model architectures during aggregation.<n>Our experimental results demonstrate that FedADP significantly outperforms existing methods, such as FlexiFed, achieving an accuracy improvement of up to 23.30%.
  arXiv  Detail & Related papers  (2025-05-10T02:57:07Z)
• Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clients [60.22876915395139]
  This paper studies a challenging robust federated learning task with model heterogeneous and data corrupted clients.<n>Data corruption is unavoidable due to factors such as random noise, compression artifacts, or environmental conditions in real-world deployment.<n>We propose a novel Robust Asymmetric Heterogeneous Federated Learning framework to address these issues.
  arXiv  Detail & Related papers  (2025-03-12T09:52:04Z)
• Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architectures [23.45003063073793]
  Federated learning (FL) enables collaborative fine-tuning of large language models without accessing raw data.<n>Existing approaches to federated fine-tuning usually adopt a uniform model architecture.<n>We propose FedAMoLE, a lightweight personalized FL framework that enables data-driven heterogeneous model architectures.
  arXiv  Detail & Related papers  (2024-11-28T13:20:38Z)
• FedPAE: Peer-Adaptive Ensemble Learning for Asynchronous and Model-Heterogeneous Federated Learning [9.084674176224109]
  Federated learning (FL) enables multiple clients with distributed data sources to collaboratively train a shared model without compromising data privacy. We introduce Federated Peer-Adaptive Ensemble Learning (FedPAE), a fully decentralized pFL algorithm that supports model heterogeneity and asynchronous learning. Our approach utilizes a peer-to-peer model sharing mechanism and ensemble selection to achieve a more refined balance between local and global information.
  arXiv  Detail & Related papers  (2024-10-17T22:47:19Z)
• Balancing Similarity and Complementarity for Federated Learning [91.65503655796603]
  Federated Learning (FL) is increasingly important in mobile and IoT systems. One key challenge in FL is managing statistical heterogeneity, such as non-i.i.d. data. We introduce a novel framework, textttFedSaC, which balances similarity and complementarity in FL cooperation.
  arXiv  Detail & Related papers  (2024-05-16T08:16:19Z)
• ARIA: On the Interaction Between Architectures, Initialization and Aggregation Methods for Federated Visual Classification [1.4585326585979703]
  We conduct the first joint ARchitecture-Initialization-Aggregation study and benchmark ARIAs across a range of medical image classification tasks. We find that, contrary to current practices, ARIA elements have to be chosen together to achieve the best possible performance. Our results shed light on good choices for each element depending on the task, the effect of normalisation layers, and the utility of SSL pre-training.
  arXiv  Detail & Related papers  (2023-11-24T17:40:31Z)
• Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learning [22.310090483499035]
  Federated learning (FL) enables edge-devices to collaboratively learn a model without disclosing their private data to a central aggregating server. Most existing FL algorithms require models of identical architecture to be deployed across the clients and server. We propose a novel ensemble knowledge transfer method named Fed-ET in which small models are trained on clients, and used to train a larger model at the server.
  arXiv  Detail & Related papers  (2022-04-27T05:18:32Z)
• Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization [107.72786199113183]
  Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data. In this paper, we propose a novel Split-Mix FL strategy for heterogeneous participants that, once training is done, provides in-situ customization of model sizes and robustness.
  arXiv  Detail & Related papers  (2022-03-18T04:58:34Z)
• Coordination Among Neural Modules Through a Shared Global Workspace [78.08062292790109]
  In cognitive science, a global workspace architecture has been proposed in which functionally specialized components share information. We show that capacity limitations have a rational basis in that they encourage specialization and compositionality.
  arXiv  Detail & Related papers  (2021-03-01T18:43:48Z)
• Edge-assisted Democratized Learning Towards Federated Analytics [67.44078999945722]
  We show the hierarchical learning structure of the proposed edge-assisted democratized learning mechanism, namely Edge-DemLearn. We also validate Edge-DemLearn as a flexible model training mechanism to build a distributed control and aggregation methodology in regions.
  arXiv  Detail & Related papers  (2020-12-01T11:46:03Z)
• HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients [42.365530133003816]
  We propose a new federated learning framework named HeteroFL to address heterogeneous clients equipped with very different computation and communication capabilities. Our solution can enable the training of heterogeneous local models with varying complexities. We show that adaptively distributing data according to clients' capabilities is both computation and communication efficient.
  arXiv  Detail & Related papers  (2020-10-03T02:55:33Z)
• Federated Residual Learning [53.77128418049985]
  We study a new form of federated learning where the clients train personalized local models and make predictions jointly with the server-side shared model. Using this new federated learning framework, the complexity of the central shared model can be minimized while still gaining all the performance benefits that joint training provides.
  arXiv  Detail & Related papers  (2020-03-28T19:55:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.