Vertical Federated Knowledge Transfer via Representation Distillation for Healthcare Collaboration Networks

• URL: http://arxiv.org/abs/2302.05675v1
• Date: Sat, 11 Feb 2023 12:15:37 GMT
• Title: Vertical Federated Knowledge Transfer via Representation Distillation for Healthcare Collaboration Networks
• Authors: Chung-ju Huang and Leye Wang and Xiao Han
• Abstract summary: We propose a unified framework for vertical federated knowledge transfer mechanism (VFedTrans) based on a novel cross-hospital representation distillation component. For each hospital, we learn a local-representation-distilled module, which can transfer the knowledge from shared samples' federated representations to enrich local samples' representations. Experiments on real-life medical datasets verify the knowledge transfer effectiveness of our framework.
• Score: 9.121410198690088
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Collaboration between healthcare institutions can significantly lessen the imbalance in medical resources across various geographic areas. However, directly sharing diagnostic information between institutions is typically not permitted due to the protection of patients' highly sensitive privacy. As a novel privacy-preserving machine learning paradigm, federated learning (FL) makes it possible to maximize the data utility among multiple medical institutions. These feature-enrichment FL techniques are referred to as vertical FL (VFL). Traditional VFL can only benefit multi-parties' shared samples, which strongly restricts its application scope. In order to improve the information-sharing capability and innovation of various healthcare-related institutions, and then to establish a next-generation open medical collaboration network, we propose a unified framework for vertical federated knowledge transfer mechanism (VFedTrans) based on a novel cross-hospital representation distillation component. Specifically, our framework includes three steps. First, shared samples' federated representations are extracted by collaboratively modeling multi-parties' joint features with current efficient vertical federated representation learning methods. Second, for each hospital, we learn a local-representation-distilled module, which can transfer the knowledge from shared samples' federated representations to enrich local samples' representations. Finally, each hospital can leverage local samples' representations enriched by the distillation module to boost arbitrary downstream machine learning tasks. The experiments on real-life medical datasets verify the knowledge transfer effectiveness of our framework.

Related papers

• UniTrans: A Unified Vertical Federated Knowledge Transfer Framework for Enhancing Cross-Hospital Collaboration [23.16178059724539]
  Cross-hospital collaboration has the potential to address disparities in medical resources across different regions. Strict privacy regulations prohibit the direct sharing of sensitive patient information between hospitals.
  arXiv  Detail & Related papers  (2025-01-20T10:33:07Z)
• FedMetaMed: Federated Meta-Learning for Personalized Medication in Distributed Healthcare Systems [7.32609591220333]
  We introduce Federated Meta-Learning for Personalized Medication (FedMetaMed) FedMetaMed combines federated learning and meta-learning to create models that adapt to diverse patient data across healthcare systems. We show that FedMetaMed outperforms state-of-the-art FL methods, showing superior generalization even on out-of-the-art cohorts.
  arXiv  Detail & Related papers  (2024-12-05T03:36:55Z)
• FedMM: Federated Multi-Modal Learning with Modality Heterogeneity in Computational Pathology [3.802258033231335]
  Federated Multi-Modal (FedMM) is a learning framework that trains multiple single-modal feature extractors to enhance subsequent classification performance. FedMM notably outperforms two baselines in accuracy and AUC metrics.
  arXiv  Detail & Related papers  (2024-02-24T16:58:42Z)
• Collaborative Training of Medical Artificial Intelligence Models with non-uniform Labels [0.07176066267895696]
  Building powerful and robust deep learning models requires training with large multi-party datasets. We propose flexible federated learning (FFL) for collaborative training on such data. We demonstrate that having heterogeneously labeled datasets, FFL-based training leads to significant performance increase.
  arXiv  Detail & Related papers  (2022-11-24T13:48:54Z)
• Federated Learning with Privacy-Preserving Ensemble Attention Distillation [63.39442596910485]
  Federated Learning (FL) is a machine learning paradigm where many local nodes collaboratively train a central model while keeping the training data decentralized. We propose a privacy-preserving FL framework leveraging unlabeled public data for one-way offline knowledge distillation. Our technique uses decentralized and heterogeneous local data like existing FL approaches, but more importantly, it significantly reduces the risk of privacy leakage.
  arXiv  Detail & Related papers  (2022-10-16T06:44:46Z)
• MS Lesion Segmentation: Revisiting Weighting Mechanisms for Federated Learning [92.91544082745196]
  Federated learning (FL) has been widely employed for medical image analysis. FL's performance is limited for multiple sclerosis (MS) lesion segmentation tasks. We propose the first FL MS lesion segmentation framework via two effective re-weighting mechanisms.
  arXiv  Detail & Related papers  (2022-05-03T14:06:03Z)
• Federated Contrastive Learning for Volumetric Medical Image Segmentation [16.3860181959878]
  Federated learning (FL) can help in this regard by learning a shared model while keeping training data local for privacy. Traditional FL requires fully-labeled data for training, which is inconvenient or sometimes infeasible to obtain. In this work, we propose an FCL framework for volumetric medical image segmentation with limited annotations.
  arXiv  Detail & Related papers  (2022-04-23T03:47:23Z)
• Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentation [66.44449514373746]
  Cross-silo federated learning (FL) has attracted much attention in medical imaging analysis with deep learning in recent years. There can be a gap between the model trained from FL and one from centralized training. We propose a novel training framework FedSM to avoid client issue and successfully close the drift gap.
  arXiv  Detail & Related papers  (2022-03-18T19:50:07Z)
• Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phases [57.90226879210227]
  FedCy is a semi-supervised learning (FSSL) method that combines FL and self-supervised learning to exploit a decentralized dataset of both labeled and unlabeled videos. We demonstrate significant performance gains over state-of-the-art FSSL methods on the task of automatic recognition of surgical phases.
  arXiv  Detail & Related papers  (2022-03-14T17:44:53Z)
• Federated Semi-supervised Medical Image Classification via Inter-client Relation Matching [58.26619456972598]
  Federated learning (FL) has emerged with increasing popularity to collaborate distributed medical institutions for training deep networks. This paper studies a practical yet challenging FL problem, named textitFederated Semi-supervised Learning (FSSL) We present a novel approach for this problem, which improves over traditional consistency regularization mechanism with a new inter-client relation matching scheme.
  arXiv  Detail & Related papers  (2021-06-16T07:58:00Z)
• FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space [63.43592895652803]
  Federated learning allows distributed medical institutions to collaboratively learn a shared prediction model with privacy protection. While at clinical deployment, the models trained in federated learning can still suffer from performance drop when applied to completely unseen hospitals outside the federation. We present a novel approach, named as Episodic Learning in Continuous Frequency Space (ELCFS), for this problem. The effectiveness of our method is demonstrated with superior performance over state-of-the-arts and in-depth ablation experiments on two medical image segmentation tasks.
  arXiv  Detail & Related papers  (2021-03-10T13:05:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.