Learning Unlabeled Clients Divergence for Federated Semi-Supervised Learning via Anchor Model Aggregation

• URL: http://arxiv.org/abs/2407.10327v2
• Date: Fri, 25 Oct 2024 14:39:37 GMT
• Title: Learning Unlabeled Clients Divergence for Federated Semi-Supervised Learning via Anchor Model Aggregation
• Authors: Marawan Elbatel, Hualiang Wang, Jixiang Chen, Hao Wang, Xiaomeng Li,
• Abstract summary: SemiAnAgg learns unlabeled client contributions via an anchor model. SemiAnAgg achieves new state-of-the-art results on four widely used FedSemi benchmarks.
• Score: 10.282711631100845
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated semi-supervised learning (FedSemi) refers to scenarios where there may be clients with fully labeled data, clients with partially labeled, and even fully unlabeled clients while preserving data privacy. However, challenges arise from client drift due to undefined heterogeneous class distributions and erroneous pseudo-labels. Existing FedSemi methods typically fail to aggregate models from unlabeled clients due to their inherent unreliability, thus overlooking unique information from their heterogeneous data distribution, leading to sub-optimal results. In this paper, we enable unlabeled client aggregation through SemiAnAgg, a novel Semi-supervised Anchor-Based federated Aggregation. SemiAnAgg learns unlabeled client contributions via an anchor model, effectively harnessing their informative value. Our key idea is that by feeding local client data to the same global model and the same consistently initialized anchor model (i.e., random model), we can measure the importance of each unlabeled client accordingly. Extensive experiments demonstrate that SemiAnAgg achieves new state-of-the-art results on four widely used FedSemi benchmarks, leading to substantial performance improvements: a 9% increase in accuracy on CIFAR-100 and a 7.6% improvement in recall on the medical dataset ISIC-18, compared with prior state-of-the-art. Code is available at: https://github.com/xmed-lab/SemiAnAgg.

Related papers

• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• Federated Learning with Only Positive Labels by Exploring Label Correlations [78.59613150221597]
  Federated learning aims to collaboratively learn a model by using the data from multiple users under privacy constraints. In this paper, we study the multi-label classification problem under the federated learning setting. We propose a novel and generic method termed Federated Averaging by exploring Label Correlations (FedALC)
  arXiv  Detail & Related papers  (2024-04-24T02:22:50Z)
• Rethinking Semi-Supervised Federated Learning: How to co-train fully-labeled and fully-unlabeled client imaging data [6.322831694506287]
  Isolated Federated Learning (IsoFed) is a learning scheme specifically designed for semi-supervised federated learning (SSFL) We propose a novel learning scheme specifically designed for SSFL that circumvents the problem by avoiding simple averaging of supervised and semi-supervised models together. In particular, our training approach consists of two parts - (a) isolated aggregation of labeled and unlabeled client models, and (b) local self-supervised pretraining of isolated global models in all clients.
  arXiv  Detail & Related papers  (2023-10-28T20:41:41Z)
• Navigating Data Heterogeneity in Federated Learning A Semi-Supervised Federated Object Detection [3.7398615061365206]
  Federated Learning (FL) has emerged as a potent framework for training models across distributed data sources. It faces challenges with limited high-quality labels and non-IID client data, particularly in applications like autonomous driving. We present a pioneering SSFOD framework, designed for scenarios where labeled data reside only at the server while clients possess unlabeled data.
  arXiv  Detail & Related papers  (2023-10-26T01:40:28Z)
• FedIL: Federated Incremental Learning from Decentralized Unlabeled Data with Convergence Analysis [23.70951896315126]
  This work considers the server with a small labeled dataset and intends to use unlabeled data in multiple clients for semi-supervised learning. We propose a new framework with a generalized model, Federated Incremental Learning (FedIL), to address the problem of how to utilize labeled data in the server and unlabeled data in clients separately.
  arXiv  Detail & Related papers  (2023-02-23T07:12:12Z)
• Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks [65.34113135080105]
  We show that not only the issue of data heterogeneity in current setups is not necessarily a problem but also in fact it can be beneficial for the FL participants. Our observations are intuitive. Our code is available at https://github.com/MMorafah/FL-SC-NIID.
  arXiv  Detail & Related papers  (2022-09-30T17:15:19Z)
• Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction [76.26710990597498]
  We show that the class-imbalance of the grouped data from randomly selected clients can lead to significant performance degradation. Based on our key observation, we design an efficient client sampling mechanism, i.e., Federated Class-balanced Sampling (Fed-CBS) In particular, we propose a measure of class-imbalance and then employ homomorphic encryption to derive this measure in a privacy-preserving way.
  arXiv  Detail & Related papers  (2022-09-30T05:42:56Z)
• RSCFed: Random Sampling Consensus Federated Semi-supervised Learning [40.998176838813045]
  Federated semi-supervised learning (FSSL) aims to derive a global model by training fully-labeled and fully-unlabeled clients or training partially labeled clients. We present a Random Sampling Consensus Federated learning, namely RSCFed, by considering the uneven reliability among models from fully-labeled clients, fully-unlabeled clients or partially labeled clients.
  arXiv  Detail & Related papers  (2022-03-26T05:10:44Z)
• SemiFed: Semi-supervised Federated Learning with Consistency and Pseudo-Labeling [14.737638416823772]
  Federated learning enables multiple clients, such as mobile phones and organizations, to collaboratively learn a shared model for prediction. In this work, we focus on a new scenario for cross-silo federated learning, where data samples of each client are partially labeled. We propose a new framework dubbed SemiFed that unifies two dominant approaches for semi-supervised learning: consistency regularization and pseudo-labeling.
  arXiv  Detail & Related papers  (2021-08-21T01:14:27Z)
• Towards Fair Federated Learning with Zero-Shot Data Augmentation [123.37082242750866]
  Federated learning has emerged as an important distributed learning paradigm, where a server aggregates a global model from many client-trained models while having no access to the client data. We propose a novel federated learning system that employs zero-shot data augmentation on under-represented data to mitigate statistical heterogeneity and encourage more uniform accuracy performance across clients in federated networks. We study two variants of this scheme, Fed-ZDAC (federated learning with zero-shot data augmentation at the clients) and Fed-ZDAS (federated learning with zero-shot data augmentation at the server).
  arXiv  Detail & Related papers  (2021-04-27T18:23:54Z)
• Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning [78.88007892742438]
  We study two essential scenarios of Federated Semi-Supervised Learning (FSSL) based on the location of the labeled data. We propose a novel method to tackle the problems, which we refer to as Federated Matching (FedMatch)
  arXiv  Detail & Related papers  (2020-06-22T09:43:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.