Communication-Efficient Hybrid Federated Learning for E-health with Horizontal and Vertical Data Partitioning

• URL: http://arxiv.org/abs/2404.10110v1
• Date: Mon, 15 Apr 2024 19:45:07 GMT
• Title: Communication-Efficient Hybrid Federated Learning for E-health with Horizontal and Vertical Data Partitioning
• Authors: Chong Yu, Shuaiqi Shen, Shiqiang Wang, Kuan Zhang, Hai Zhao,
• Abstract summary: E-health allows smart devices and medical institutions to collaboratively collect patients' data, which is trained by Artificial Intelligence (AI) technologies to help doctors make diagnosis. Applying federated learning in e-health faces many challenges. Medical data is both horizontally and vertically partitioned. A naive combination of HFL and VFL has limitations including low training efficiency, unsound convergence analysis, and lack of parameter tuning strategies.
• Score: 67.49221252724229
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: E-health allows smart devices and medical institutions to collaboratively collect patients' data, which is trained by Artificial Intelligence (AI) technologies to help doctors make diagnosis. By allowing multiple devices to train models collaboratively, federated learning is a promising solution to address the communication and privacy issues in e-health. However, applying federated learning in e-health faces many challenges. First, medical data is both horizontally and vertically partitioned. Since single Horizontal Federated Learning (HFL) or Vertical Federated Learning (VFL) techniques cannot deal with both types of data partitioning, directly applying them may consume excessive communication cost due to transmitting a part of raw data when requiring high modeling accuracy. Second, a naive combination of HFL and VFL has limitations including low training efficiency, unsound convergence analysis, and lack of parameter tuning strategies. In this paper, we provide a thorough study on an effective integration of HFL and VFL, to achieve communication efficiency and overcome the above limitations when data is both horizontally and vertically partitioned. Specifically, we propose a hybrid federated learning framework with one intermediate result exchange and two aggregation phases. Based on this framework, we develop a Hybrid Stochastic Gradient Descent (HSGD) algorithm to train models. Then, we theoretically analyze the convergence upper bound of the proposed algorithm. Using the convergence results, we design adaptive strategies to adjust the training parameters and shrink the size of transmitted data. Experimental results validate that the proposed HSGD algorithm can achieve the desired accuracy while reducing communication cost, and they also verify the effectiveness of the adaptive strategies.

Related papers

• FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data [54.81695390763957]
  Federated learning is an emerging distributed machine learning method. We propose a heterogeneous local variant of AMSGrad, named FedLALR, in which each client adjusts its learning rate. We show that our client-specified auto-tuned learning rate scheduling can converge and achieve linear speedup with respect to the number of clients.
  arXiv  Detail & Related papers  (2023-09-18T12:35:05Z)
• Vertical Federated Learning over Cloud-RAN: Convergence Analysis and System Optimization [82.12796238714589]
  We propose a novel cloud radio access network (Cloud-RAN) based vertical FL system to enable fast and accurate model aggregation. We characterize the convergence behavior of the vertical FL algorithm considering both uplink and downlink transmissions. We establish a system optimization framework by joint transceiver and fronthaul quantization design, for which successive convex approximation and alternate convex search based system optimization algorithms are developed.
  arXiv  Detail & Related papers  (2023-05-04T09:26:03Z)
• Exploratory Analysis of Federated Learning Methods with Differential Privacy on MIMIC-III [0.7349727826230862]
  Federated learning methods offer the possibility of training machine learning models on privacy-sensitive data sets. We present an evaluation of the impact of different federation and differential privacy techniques when training models on the open-source MIMIC-III dataset.
  arXiv  Detail & Related papers  (2023-02-08T17:27:44Z)
• Distributed Contrastive Learning for Medical Image Segmentation [16.3860181959878]
  Supervised deep learning needs a large amount of labeled data to achieve high performance. In medical imaging analysis, each site may only have a limited amount of data and labels, which makes learning ineffective. We propose two federated self-supervised learning frameworks for medical image segmentation with limited annotations.
  arXiv  Detail & Related papers  (2022-08-07T20:47:05Z)
• Federated Offline Reinforcement Learning [55.326673977320574]
  We propose a multi-site Markov decision process model that allows for both homogeneous and heterogeneous effects across sites. We design the first federated policy optimization algorithm for offline RL with sample complexity. We give a theoretical guarantee for the proposed algorithm, where the suboptimality for the learned policies is comparable to the rate as if data is not distributed.
  arXiv  Detail & Related papers  (2022-06-11T18:03:26Z)
• Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentation [48.821062916381685]
  Federated learning (FL) is a distributed machine learning technique that enables collaborative model training while avoiding explicit data sharing. In this work, we propose an efficient reinforcement learning(RL)-based federated hyperparameter optimization algorithm, termed Auto-FedRL. The effectiveness of the proposed method is validated on a heterogeneous data split of the CIFAR-10 dataset and two real-world medical image segmentation datasets.
  arXiv  Detail & Related papers  (2022-03-12T04:11:42Z)
• Communication-Efficient Federated Learning with Compensated Overlap-FedAvg [22.636184975591004]
  Federated learning is proposed to perform model training by multiple clients' combined data without the dataset sharing within the cluster. We propose Overlap-FedAvg, a framework that parallels the model training phase with model uploading & downloading phase. Overlap-FedAvg is further developed with a hierarchical computing strategy, a data compensation mechanism and a nesterov accelerated gradients(NAG) algorithm.
  arXiv  Detail & Related papers  (2020-12-12T02:50:09Z)
• Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning [151.47900584193025]
  We propose an asynchronous federated SGD (AFSGD-VP) algorithm and its SVRG and SAGA variants on the vertically partitioned data. To the best of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first asynchronous federated learning algorithms for vertically partitioned data.
  arXiv  Detail & Related papers  (2020-08-14T08:08:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.