dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis

• URL: http://arxiv.org/abs/2503.10412v3
• Date: Wed, 19 Mar 2025 16:39:02 GMT
• Title: dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis
• Abstract summary: We propose a decentralized federated learning framework named dFLMoE.<n>In our framework, clients directly exchange lightweight head models with each other.<n>We validate our framework on multiple medical tasks, demonstrating that our method evidently outperforms state-of-the-art approaches.
• Score: 45.92342950385488
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning has wide applications in the medical field. It enables knowledge sharing among different healthcare institutes while protecting patients' privacy. However, existing federated learning systems are typically centralized, requiring clients to upload client-specific knowledge to a central server for aggregation. This centralized approach would integrate the knowledge from each client into a centralized server, and the knowledge would be already undermined during the centralized integration before it reaches back to each client. Besides, the centralized approach also creates a dependency on the central server, which may affect training stability if the server malfunctions or connections are unstable. To address these issues, we propose a decentralized federated learning framework named dFLMoE. In our framework, clients directly exchange lightweight head models with each other. After exchanging, each client treats both local and received head models as individual experts, and utilizes a client-specific Mixture of Experts (MoE) approach to make collective decisions. This design not only reduces the knowledge damage with client-specific aggregations but also removes the dependency on the central server to enhance the robustness of the framework. We validate our framework on multiple medical tasks, demonstrating that our method evidently outperforms state-of-the-art approaches under both model homogeneity and heterogeneity settings.

Related papers

• Distributed Federated Learning by Alternating Periods of Training [0.0]
  Federated learning is a privacy-focused approach towards machine learning where models are trained on client devices with locally available data and aggregated at a central server.<n>We present a distributed approach to federated learning comprising multiple servers with inter-server communication capabilities.<n>We propose a novel DFL (Distributed Federated Learning) algorithm which uses alternating periods of local training on the client data followed by global training among servers.
  arXiv  Detail & Related papers  (2026-01-05T05:06:58Z)
• Federated Client-tailored Adapter for Medical Image Segmentation [21.964553228831427]
  We propose a novel Federated Client-tailored Adapter (FCA) framework for medical image segmentation. FCA achieves stable and client-tailored adaptive segmentation without sharing sensitive local data. We develop two client-tailored federated updating strategies that adaptively decompose the adapter into common and individual components.
  arXiv  Detail & Related papers  (2025-04-25T02:20:25Z)
• FedSPD: A Soft-clustering Approach for Personalized Decentralized Federated Learning [18.38030098837294]
  Federated learning is a framework for distributed clients to collaboratively train a machine learning model using local data. We propose FedSPD, an efficient personalized federated learning algorithm for the decentralized setting. We show that FedSPD learns accurate models even in low-connectivity networks.
  arXiv  Detail & Related papers  (2024-10-24T15:48:34Z)
• Brain Storm Optimization Based Swarm Learning for Diabetic Retinopathy Image Classification [5.440545944342685]
  This paper integrates the brain storm optimization algorithm into the swarm learning framework, named BSO-SL. The proposed method has been validated on a real-world diabetic retinopathy image classification dataset.
  arXiv  Detail & Related papers  (2024-04-24T01:37:20Z)
• Blockchain-enabled Trustworthy Federated Unlearning [50.01101423318312]
  Federated unlearning is a promising paradigm for protecting the data ownership of distributed clients. Existing works require central servers to retain the historical model parameters from distributed clients. This paper proposes a new blockchain-enabled trustworthy federated unlearning framework.
  arXiv  Detail & Related papers  (2024-01-29T07:04:48Z)
• Scalable Collaborative Learning via Representation Sharing [53.047460465980144]
  Federated learning (FL) and Split Learning (SL) are two frameworks that enable collaborative learning while keeping the data private (on device) In FL, each data holder trains a model locally and releases it to a central server for aggregation. In SL, the clients must release individual cut-layer activations (smashed data) to the server and wait for its response (during both inference and back propagation). In this work, we present a novel approach for privacy-preserving machine learning, where the clients collaborate via online knowledge distillation using a contrastive loss.
  arXiv  Detail & Related papers  (2022-11-20T10:49:22Z)
• Exploring Semantic Attributes from A Foundation Model for Federated Learning of Disjoint Label Spaces [46.59992662412557]
  In this work, we consider transferring mid-level semantic knowledge (such as attribute) which is not sensitive to specific objects of interest. We formulate a new Federated Zero-Shot Learning (FZSL) paradigm to learn mid-level semantic knowledge at multiple local clients. To improve model discriminative ability, we propose to explore semantic knowledge augmentation from external knowledge.
  arXiv  Detail & Related papers  (2022-08-29T10:05:49Z)
• DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training [84.81043932706375]
  We propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL. Dis-PFL employs personalized sparse masks to customize sparse local models on the edge. We demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities.
  arXiv  Detail & Related papers  (2022-06-01T02:20:57Z)
• Implicit Model Specialization through DAG-based Decentralized Federated Learning [0.0]
  Federated learning allows a group of distributed clients to train a common machine learning model on private data. We propose a unified approach to decentralization and personalization in federated learning. Our evaluation shows that the specialization of models emerges directly from the DAG-based communication of model updates.
  arXiv  Detail & Related papers  (2021-11-01T20:55:47Z)
• DeFed: A Principled Decentralized and Privacy-Preserving Federated Learning Algorithm [10.487593244018933]
  Federated learning enables a large number of clients to participate in learning a shared model while maintaining the training data stored in each client. Here we propose a principled decentralized federated learning algorithm (DeFed), which removes the central client in the classical Federated Averaging (FedAvg) setting. The proposed algorithm is proven to reach the global minimum with a convergence rate of $O(1/T)$ when the loss function is smooth and strongly convex, where $T$ is the number of iterations in gradient descent.
  arXiv  Detail & Related papers  (2021-07-15T07:39:19Z)
• Decentralized Federated Learning via Mutual Knowledge Transfer [37.5341683644709]
  Decentralized federated learning (DFL) is a problem in the Internet of things (IoT) systems. We propose a mutual knowledge transfer (Def-KT) algorithm where local clients fuse models by transferring their learnt knowledge to each other. Our experiments on the MNIST, Fashion-MNIST, and CIFAR10 datasets reveal datasets that the proposed Def-KT algorithm significantly outperforms the baseline DFL methods.
  arXiv  Detail & Related papers  (2020-12-24T01:43:53Z)
• Decentralised Learning from Independent Multi-Domain Labels for Person Re-Identification [69.29602103582782]
  Deep learning has been successful for many computer vision tasks due to the availability of shared and centralised large-scale training data. However, increasing awareness of privacy concerns poses new challenges to deep learning, especially for person re-identification (Re-ID) We propose a novel paradigm called Federated Person Re-Identification (FedReID) to construct a generalisable global model (a central server) by simultaneously learning with multiple privacy-preserved local models (local clients) This client-server collaborative learning process is iteratively performed under privacy control, enabling FedReID to realise decentralised learning without sharing distributed data nor collecting any
  arXiv  Detail & Related papers  (2020-06-07T13:32:33Z)
• Multi-Center Federated Learning [62.57229809407692]
  This paper proposes a novel multi-center aggregation mechanism for federated learning. It learns multiple global models from the non-IID user data and simultaneously derives the optimal matching between users and centers. Our experimental results on benchmark datasets show that our method outperforms several popular federated learning methods.
  arXiv  Detail & Related papers  (2020-05-03T09:14:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.