Federated Semi-Supervised Learning with Annotation Heterogeneity

• URL: http://arxiv.org/abs/2303.02445v1
• Date: Sat, 4 Mar 2023 16:04:49 GMT
• Title: Federated Semi-Supervised Learning with Annotation Heterogeneity
• Authors: Xinyi Shang, Gang Huang, Yang Lu, Jian Lou, Bo Han, Yiu-ming Cheung, Hanzi Wang
• Abstract summary: We propose a novel framework called Heterogeneously Annotated Semi-Supervised LEarning (HASSLE) It is a dual-model framework with two models trained separately on labeled and unlabeled data. The dual models can implicitly learn from both types of data across different clients, although each dual model is only trained locally on a single type of data.
• Score: 57.12560313403097
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated Semi-Supervised Learning (FSSL) aims to learn a global model from different clients in an environment with both labeled and unlabeled data. Most of the existing FSSL work generally assumes that both types of data are available on each client. In this paper, we study a more general problem setup of FSSL with annotation heterogeneity, where each client can hold an arbitrary percentage (0%-100%) of labeled data. To this end, we propose a novel FSSL framework called Heterogeneously Annotated Semi-Supervised LEarning (HASSLE). Specifically, it is a dual-model framework with two models trained separately on labeled and unlabeled data such that it can be simply applied to a client with an arbitrary labeling percentage. Furthermore, a mutual learning strategy called Supervised-Unsupervised Mutual Alignment (SUMA) is proposed for the dual models within HASSLE with global residual alignment and model proximity alignment. Subsequently, the dual models can implicitly learn from both types of data across different clients, although each dual model is only trained locally on a single type of data. Experiments verify that the dual models in HASSLE learned by SUMA can mutually learn from each other, thereby effectively utilizing the information of both types of data across different clients.

Related papers

• Combating Data Imbalances in Federated Semi-supervised Learning with Dual Regulators [40.12377870379059]
  Federated semi-supervised learning (FSSL) emerges to train models from a small fraction of labeled data. We propose a novel FSSL framework with dual regulators, FedDure. We show that FedDure is superior to the existing methods across a wide range of settings.
  arXiv  Detail & Related papers  (2023-07-11T15:45:03Z)
• Dual Class-Aware Contrastive Federated Semi-Supervised Learning [9.742389743497045]
  We present a novel Federated Semi-Supervised Learning (FSSL) method called Dual Class-aware Contrastive Federated Semi-Supervised Learning (DCCFSSL) By implementing a dual class-aware contrastive module, DCCFSSL establishes a unified training objective for different clients to tackle large deviations. Our experiments show that DCCFSSL outperforms current state-of-the-art methods on three benchmark datasets.
  arXiv  Detail & Related papers  (2022-11-16T13:54:31Z)
• Optimizing Server-side Aggregation For Robust Federated Learning via Subspace Training [80.03567604524268]
  Non-IID data distribution across clients and poisoning attacks are two main challenges in real-world federated learning systems. We propose SmartFL, a generic approach that optimize the server-side aggregation process. We provide theoretical analyses of the convergence and generalization capacity for SmartFL.
  arXiv  Detail & Related papers  (2022-11-10T13:20:56Z)
• Federated Learning from Only Unlabeled Data with Class-Conditional-Sharing Clients [98.22390453672499]
  Supervised federated learning (FL) enables multiple clients to share the trained model without sharing their labeled data. We propose federation of unsupervised learning (FedUL), where the unlabeled data are transformed into surrogate labeled data for each of the clients.
  arXiv  Detail & Related papers  (2022-04-07T09:12:00Z)
• SemiFed: Semi-supervised Federated Learning with Consistency and Pseudo-Labeling [14.737638416823772]
  Federated learning enables multiple clients, such as mobile phones and organizations, to collaboratively learn a shared model for prediction. In this work, we focus on a new scenario for cross-silo federated learning, where data samples of each client are partially labeled. We propose a new framework dubbed SemiFed that unifies two dominant approaches for semi-supervised learning: consistency regularization and pseudo-labeling.
  arXiv  Detail & Related papers  (2021-08-21T01:14:27Z)
• Unifying Distillation with Personalization in Federated Learning [1.8262547855491458]
  Federated learning (FL) is a decentralized privacy-preserving learning technique in which clients learn a joint collaborative model through a central aggregator without sharing their data. In this setting, all clients learn a single common predictor (FedAvg), which does not generalize well on each client's local data due to the statistical data heterogeneity among clients. In this paper, we address this problem with PersFL, a two-stage personalized learning algorithm. In the first stage, PersFL finds the optimal teacher model of each client during the FL training phase. In the second stage, PersFL distills the useful knowledge from
  arXiv  Detail & Related papers  (2021-05-31T17:54:29Z)
• Federated Unsupervised Representation Learning [56.715917111878106]
  We formulate a new problem in federated learning called Federated Unsupervised Representation Learning (FURL) to learn a common representation model without supervision. FedCA is composed of two key modules: dictionary module to aggregate the representations of samples from each client and share with all clients for consistency of representation space and alignment module to align the representation of each client on a base model trained on a public data.
  arXiv  Detail & Related papers  (2020-10-18T13:28:30Z)
• Federated Mutual Learning [65.46254760557073]
  Federated Mutual Leaning (FML) allows clients training a generalized model collaboratively and a personalized model independently. The experiments show that FML can achieve better performance than alternatives in typical Federated learning setting.
  arXiv  Detail & Related papers  (2020-06-27T09:35:03Z)
• Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning [78.88007892742438]
  We study two essential scenarios of Federated Semi-Supervised Learning (FSSL) based on the location of the labeled data. We propose a novel method to tackle the problems, which we refer to as Federated Matching (FedMatch)
  arXiv  Detail & Related papers  (2020-06-22T09:43:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.