Robust Split Federated Learning for U-shaped Medical Image Networks

• URL: http://arxiv.org/abs/2212.06378v1
• Date: Tue, 13 Dec 2022 05:26:31 GMT
• Title: Robust Split Federated Learning for U-shaped Medical Image Networks
• Authors: Ziyuan Yang, Yingyu Chen, Huijie Huangfu, Maosong Ran, Hui Wang, Xiaoxiao Li and Yi Zhang
• Abstract summary: We propose Robust Split Federated Learning (RoS-FL) for U-shaped medical image networks. RoS-FL is a novel hybrid learning paradigm of Federated Learning (FL) and Split Learning (SL)
• Score: 16.046153872932653
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: U-shaped networks are widely used in various medical image tasks, such as segmentation, restoration and reconstruction, but most of them usually rely on centralized learning and thus ignore privacy issues. To address the privacy concerns, federated learning (FL) and split learning (SL) have attracted increasing attention. However, it is hard for both FL and SL to balance the local computational cost, model privacy and parallel training simultaneously. To achieve this goal, in this paper, we propose Robust Split Federated Learning (RoS-FL) for U-shaped medical image networks, which is a novel hybrid learning paradigm of FL and SL. Previous works cannot preserve the data privacy, including the input, model parameters, label and output simultaneously. To effectively deal with all of them, we design a novel splitting method for U-shaped medical image networks, which splits the network into three parts hosted by different parties. Besides, the distributed learning methods usually suffer from a drift between local and global models caused by data heterogeneity. Based on this consideration, we propose a dynamic weight correction strategy (\textbf{DWCS}) to stabilize the training process and avoid model drift. Specifically, a weight correction loss is designed to quantify the drift between the models from two adjacent communication rounds. By minimizing this loss, a correction model is obtained. Then we treat the weighted sum of correction model and final round models as the result. The effectiveness of the proposed RoS-FL is supported by extensive experimental results on different tasks. Related codes will be released at https://github.com/Zi-YuanYang/RoS-FL.

Related papers

• R-SFLLM: Jamming Resilient Framework for Split Federated Learning with Large Language Models [83.77114091471822]
  Split federated learning (SFL) is a compute-efficient paradigm in distributed machine learning (ML) A challenge in SFL, particularly when deployed over wireless channels, is the susceptibility of transmitted model parameters to adversarial jamming. This is particularly pronounced for word embedding parameters in large language models (LLMs), which are crucial for language understanding. A physical layer framework is developed for resilient SFL with LLMs (R-SFLLM) over wireless networks.
  arXiv  Detail & Related papers  (2024-07-16T12:21:29Z)
• Harnessing Increased Client Participation with Cohort-Parallel Federated Learning [2.9593087583214173]
  Federated Learning (FL) is a machine learning approach where nodes collaboratively train a global model. We introduce Cohort-Parallel Federated Learning (CPFL), a novel learning approach where each cohort independently trains a global model. CPFL with four cohorts, non-IID data distribution, and CIFAR-10 yields a 1.9$times$ reduction in train time and a 1.3$times$ reduction in resource usage.
  arXiv  Detail & Related papers  (2024-05-24T15:34:09Z)
• Heterogeneous Federated Learning with Splited Language Model [22.65325348176366]
  Federated Split Learning (FSL) is a promising distributed learning paradigm in practice. In this paper, we harness Pre-trained Image Transformers (PITs) as the initial model, coined FedV, to accelerate the training process and improve model robustness. We are the first to provide a systematic evaluation of FSL methods with PITs in real-world datasets, different partial device participations, and heterogeneous data splits.
  arXiv  Detail & Related papers  (2024-03-24T07:33:08Z)
• Adaptive Model Pruning and Personalization for Federated Learning over Wireless Networks [72.59891661768177]
  Federated learning (FL) enables distributed learning across edge devices while protecting data privacy. We consider a FL framework with partial model pruning and personalization to overcome these challenges. This framework splits the learning model into a global part with model pruning shared with all devices to learn data representations and a personalized part to be fine-tuned for a specific device.
  arXiv  Detail & Related papers  (2023-09-04T21:10:45Z)
• NeFL: Nested Model Scaling for Federated Learning with System Heterogeneous Clients [44.89061671579694]
  Federated learning (FL) enables distributed training while preserving data privacy, but stragglers-slow or incapable clients-can significantly slow down the total training time and degrade performance. We propose nested federated learning (NeFL), a framework that efficiently divides deep neural networks into submodels using both depthwise and widthwise scaling. NeFL achieves performance gain, especially for the worst-case submodel compared to baseline approaches.
  arXiv  Detail & Related papers  (2023-08-15T13:29:14Z)
• Federated Zero-Shot Learning for Visual Recognition [55.65879596326147]
  We propose a novel Federated Zero-Shot Learning FedZSL framework. FedZSL learns a central model from the decentralized data residing on edge devices. The effectiveness and robustness of FedZSL are demonstrated by extensive experiments conducted on three zero-shot benchmark datasets.
  arXiv  Detail & Related papers  (2022-09-05T14:49:34Z)
• Distributed Contrastive Learning for Medical Image Segmentation [16.3860181959878]
  Supervised deep learning needs a large amount of labeled data to achieve high performance. In medical imaging analysis, each site may only have a limited amount of data and labels, which makes learning ineffective. We propose two federated self-supervised learning frameworks for medical image segmentation with limited annotations.
  arXiv  Detail & Related papers  (2022-08-07T20:47:05Z)
• Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization [107.72786199113183]
  Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data. In this paper, we propose a novel Split-Mix FL strategy for heterogeneous participants that, once training is done, provides in-situ customization of model sizes and robustness.
  arXiv  Detail & Related papers  (2022-03-18T04:58:34Z)
• Parallel Successive Learning for Dynamic Distributed Model Training over Heterogeneous Wireless Networks [50.68446003616802]
  Federated learning (FedL) has emerged as a popular technique for distributing model training over a set of wireless devices. We develop parallel successive learning (PSL), which expands the FedL architecture along three dimensions. Our analysis sheds light on the notion of cold vs. warmed up models, and model inertia in distributed machine learning.
  arXiv  Detail & Related papers  (2022-02-07T05:11:01Z)
• Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devices [40.69663094185572]
  Federated learning (FL) is a popular framework for training an AI model using distributed mobile data in a wireless network. One main challenge confronting practical FL is that resource constrained devices struggle with the computation intensive task of updating a deep-neural network model. To tackle the challenge, in this paper, a federated dropout (FedDrop) scheme is proposed building on the classic dropout scheme for random model pruning.
  arXiv  Detail & Related papers  (2021-09-30T16:52:13Z)
• UVeQFed: Universal Vector Quantization for Federated Learning [179.06583469293386]
  Federated learning (FL) is an emerging approach to train such learning models without requiring the users to share their possibly private labeled data. In FL, each user trains its copy of the learning model locally. The server then collects the individual updates and aggregates them into a global model. We show that combining universal vector quantization methods with FL yields a decentralized training system in which the compression of the trained models induces only a minimum distortion.
  arXiv  Detail & Related papers  (2020-06-05T07:10:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.