SelfFed: Self-supervised Federated Learning for Data Heterogeneity and
Label Scarcity in IoMT
- URL: http://arxiv.org/abs/2307.01514v1
- Date: Tue, 4 Jul 2023 06:50:16 GMT
- Title: SelfFed: Self-supervised Federated Learning for Data Heterogeneity and
Label Scarcity in IoMT
- Authors: Sunder Ali Khowaja, Kapal Dev, Syed Muhammad Anwar, Marius George
Linguraru
- Abstract summary: We propose the SelfFed framework for Internet of Medical Things (IoMT)
Our proposed SelfFed framework works in two phases. The first phase is the pre-training paradigm that performs augmentive modeling.
The second phase is the fine-tuning paradigm that introduces contrastive network and a novel aggregation strategy.
- Score: 9.925707806264613
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Self-supervised learning in federated learning paradigm has been gaining a
lot of interest both in industry and research due to the collaborative learning
capability on unlabeled yet isolated data. However, self-supervised based
federated learning strategies suffer from performance degradation due to label
scarcity and diverse data distributions, i.e., data heterogeneity. In this
paper, we propose the SelfFed framework for Internet of Medical Things (IoMT).
Our proposed SelfFed framework works in two phases. The first phase is the
pre-training paradigm that performs augmentive modeling using Swin Transformer
based encoder in a decentralized manner. The first phase of SelfFed framework
helps to overcome the data heterogeneity issue. The second phase is the
fine-tuning paradigm that introduces contrastive network and a novel
aggregation strategy that is trained on limited labeled data for a target task
in a decentralized manner. This fine-tuning stage overcomes the label scarcity
problem. We perform our experimental analysis on publicly available medical
imaging datasets and show that our proposed SelfFed framework performs better
when compared to existing baselines concerning non-independent and identically
distributed (IID) data and label scarcity. Our method achieves a maximum
improvement of 8.8% and 4.1% on Retina and COVID-FL datasets on non-IID
dataset. Further, our proposed method outperforms existing baselines even when
trained on a few (10%) labeled instances.
Related papers
- Investigation of Federated Learning Algorithms for Retinal Optical
Coherence Tomography Image Classification with Statistical Heterogeneity [6.318288071829899]
We investigate the effectiveness of FedAvg and FedProx to train an OCT image classification model in a decentralized fashion.
We partitioned a publicly available OCT dataset across multiple clients under IID and Non-IID settings and conducted local training on the subsets for each client.
arXiv Detail & Related papers (2024-02-15T15:58:42Z) - Fake It Till Make It: Federated Learning with Consensus-Oriented
Generation [52.82176415223988]
We propose federated learning with consensus-oriented generation (FedCOG)
FedCOG consists of two key components at the client side: complementary data generation and knowledge-distillation-based model training.
Experiments on classical and real-world FL datasets show that FedCOG consistently outperforms state-of-the-art methods.
arXiv Detail & Related papers (2023-12-10T18:49:59Z) - Navigating Data Heterogeneity in Federated Learning A Semi-Supervised
Federated Object Detection [3.7398615061365206]
Federated Learning (FL) has emerged as a potent framework for training models across distributed data sources.
It faces challenges with limited high-quality labels and non-IID client data, particularly in applications like autonomous driving.
We present a pioneering SSFOD framework, designed for scenarios where labeled data reside only at the server while clients possess unlabeled data.
arXiv Detail & Related papers (2023-10-26T01:40:28Z) - Benchmarking FedAvg and FedCurv for Image Classification Tasks [1.376408511310322]
This paper focuses on the problem of statistical heterogeneity of the data in the same federated network.
Several Federated Learning algorithms, such as FedAvg, FedProx and Federated Curvature (FedCurv) have already been proposed.
As a side product of this work, we release the non-IID version of the datasets we used so to facilitate further comparisons from the FL community.
arXiv Detail & Related papers (2023-03-31T10:13:01Z) - Cluster-level pseudo-labelling for source-free cross-domain facial
expression recognition [94.56304526014875]
We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER)
Our method exploits self-supervised pretraining to learn good feature representations from the target data.
We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
arXiv Detail & Related papers (2022-10-11T08:24:50Z) - Rethinking Data Heterogeneity in Federated Learning: Introducing a New
Notion and Standard Benchmarks [65.34113135080105]
We show that not only the issue of data heterogeneity in current setups is not necessarily a problem but also in fact it can be beneficial for the FL participants.
Our observations are intuitive.
Our code is available at https://github.com/MMorafah/FL-SC-NIID.
arXiv Detail & Related papers (2022-09-30T17:15:19Z) - Label-Efficient Self-Supervised Federated Learning for Tackling Data
Heterogeneity in Medical Imaging [23.08596805950814]
We present a robust and label-efficient self-supervised FL framework for medical image analysis.
Specifically, we introduce a novel distributed self-supervised pre-training paradigm into the existing FL pipeline.
We show that our self-supervised FL algorithm generalizes well to out-of-distribution data and learns federated models more effectively in limited label scenarios.
arXiv Detail & Related papers (2022-05-17T18:33:43Z) - Local Learning Matters: Rethinking Data Heterogeneity in Federated
Learning [61.488646649045215]
Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
arXiv Detail & Related papers (2021-11-28T19:03:39Z) - Quasi-Global Momentum: Accelerating Decentralized Deep Learning on
Heterogeneous Data [77.88594632644347]
Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks.
In realistic learning scenarios, the presence of heterogeneity across different clients' local datasets poses an optimization challenge.
We propose a novel momentum-based method to mitigate this decentralized training difficulty.
arXiv Detail & Related papers (2021-02-09T11:27:14Z) - TraND: Transferable Neighborhood Discovery for Unsupervised Cross-domain
Gait Recognition [77.77786072373942]
This paper proposes a Transferable Neighborhood Discovery (TraND) framework to bridge the domain gap for unsupervised cross-domain gait recognition.
We design an end-to-end trainable approach to automatically discover the confident neighborhoods of unlabeled samples in the latent space.
Our method achieves state-of-the-art results on two public datasets, i.e., CASIA-B and OU-LP.
arXiv Detail & Related papers (2021-02-09T03:07:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.