SelfFed: Self-supervised Federated Learning for Data Heterogeneity and Label Scarcity in IoMT

• URL: http://arxiv.org/abs/2307.01514v2
• Date: Thu, 10 Oct 2024 14:27:22 GMT
• Title: SelfFed: Self-supervised Federated Learning for Data Heterogeneity and Label Scarcity in IoMT
• Authors: Sunder Ali Khowaja, Kapal Dev, Syed Muhammad Anwar, Marius George Linguraru,
• Abstract summary: We propose the SelfFed framework for Internet of Medical Things (IoMT) Our proposed SelfFed framework works in two phases. The first phase is the pre-training paradigm that performs augmentive modeling. The second phase is the fine-tuning paradigm that introduces contrastive network and a novel aggregation strategy.
• Score: 17.07904450821442
• License:
• Abstract: Self-supervised learning in federated learning paradigm has been gaining a lot of interest both in industry and research due to the collaborative learning capability on unlabeled yet isolated data. However, self-supervised based federated learning strategies suffer from performance degradation due to label scarcity and diverse data distributions, i.e., data heterogeneity. In this paper, we propose the SelfFed framework for Internet of Medical Things (IoMT). Our proposed SelfFed framework works in two phases. The first phase is the pre-training paradigm that performs augmentive modeling using Swin Transformer based encoder in a decentralized manner. The first phase of SelfFed framework helps to overcome the data heterogeneity issue. The second phase is the fine-tuning paradigm that introduces contrastive network and a novel aggregation strategy that is trained on limited labeled data for a target task in a decentralized manner. This fine-tuning stage overcomes the label scarcity problem. We perform our experimental analysis on publicly available medical imaging datasets and show that our proposed SelfFed framework performs better when compared to existing baselines concerning non-independent and identically distributed (IID) data and label scarcity. Our method achieves a maximum improvement of 8.8% and 4.1% on Retina and COVID-FL datasets on non-IID dataset. Further, our proposed method outperforms existing baselines even when trained on a few (10%) labeled instances.

Related papers

• A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning [14.466679488063217]
  One-shot federated learning (FL) limits the communication between the server and clients to a single round. We propose a unified, data-free, one-shot FL framework (FedHydra) that can effectively address both model and data heterogeneity.
  arXiv  Detail & Related papers  (2024-10-28T15:20:52Z)
• FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Models [37.76576626976729]
  One-Shot Federated Learning (OSFL), a special decentralized machine learning paradigm, has recently gained significant attention. Current methods face challenges due to client data heterogeneity and limited data quantity when applied to real-world OSFL systems. We propose Federated Bi-Level Personalization (FedBiP), which personalizes the pretrained LDM at both instance-level and concept-level.
  arXiv  Detail & Related papers  (2024-10-07T07:45:18Z)
• Fake It Till Make It: Federated Learning with Consensus-Oriented Generation [52.82176415223988]
  We propose federated learning with consensus-oriented generation (FedCOG) FedCOG consists of two key components at the client side: complementary data generation and knowledge-distillation-based model training. Experiments on classical and real-world FL datasets show that FedCOG consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-12-10T18:49:59Z)
• Benchmarking FedAvg and FedCurv for Image Classification Tasks [1.376408511310322]
  This paper focuses on the problem of statistical heterogeneity of the data in the same federated network. Several Federated Learning algorithms, such as FedAvg, FedProx and Federated Curvature (FedCurv) have already been proposed. As a side product of this work, we release the non-IID version of the datasets we used so to facilitate further comparisons from the FL community.
  arXiv  Detail & Related papers  (2023-03-31T10:13:01Z)
• Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition [94.56304526014875]
  We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER) Our method exploits self-supervised pretraining to learn good feature representations from the target data. We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
  arXiv  Detail & Related papers  (2022-10-11T08:24:50Z)
• Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks [65.34113135080105]
  We show that not only the issue of data heterogeneity in current setups is not necessarily a problem but also in fact it can be beneficial for the FL participants. Our observations are intuitive. Our code is available at https://github.com/MMorafah/FL-SC-NIID.
  arXiv  Detail & Related papers  (2022-09-30T17:15:19Z)
• Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging [23.08596805950814]
  We present a robust and label-efficient self-supervised FL framework for medical image analysis. Specifically, we introduce a novel distributed self-supervised pre-training paradigm into the existing FL pipeline. We show that our self-supervised FL algorithm generalizes well to out-of-distribution data and learns federated models more effectively in limited label scenarios.
  arXiv  Detail & Related papers  (2022-05-17T18:33:43Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data [77.88594632644347]
  Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of heterogeneity across different clients' local datasets poses an optimization challenge. We propose a novel momentum-based method to mitigate this decentralized training difficulty.
  arXiv  Detail & Related papers  (2021-02-09T11:27:14Z)
• TraND: Transferable Neighborhood Discovery for Unsupervised Cross-domain Gait Recognition [77.77786072373942]
  This paper proposes a Transferable Neighborhood Discovery (TraND) framework to bridge the domain gap for unsupervised cross-domain gait recognition. We design an end-to-end trainable approach to automatically discover the confident neighborhoods of unlabeled samples in the latent space. Our method achieves state-of-the-art results on two public datasets, i.e., CASIA-B and OU-LP.
  arXiv  Detail & Related papers  (2021-02-09T03:07:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.