FedCon: A Contrastive Framework for Federated Semi-Supervised Learning
- URL: http://arxiv.org/abs/2109.04533v1
- Date: Thu, 9 Sep 2021 19:47:21 GMT
- Title: FedCon: A Contrastive Framework for Federated Semi-Supervised Learning
- Authors: Zewei Long, Jiaqi Wang, Yaqing Wang, Houping Xiao, Fenglong Ma
- Abstract summary: Federated Semi-Supervised Learning (FedSSL) has gained rising attention from both academic and industrial researchers.
FedCon introduces a new learning paradigm, i.e., contractive learning, to FedSSL.
- Score: 26.520767887801142
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Federated Semi-Supervised Learning (FedSSL) has gained rising attention from
both academic and industrial researchers, due to its unique characteristics of
co-training machine learning models with isolated yet unlabeled data. Most
existing FedSSL methods focus on the classical scenario, i.e, the labeled and
unlabeled data are stored at the client side. However, in real world
applications, client users may not provide labels without any incentive. Thus,
the scenario of labels at the server side is more practical. Since unlabeled
data and labeled data are decoupled, most existing FedSSL approaches may fail
to deal with such a scenario. To overcome this problem, in this paper, we
propose FedCon, which introduces a new learning paradigm, i.e., contractive
learning, to FedSSL. Experimental results on three datasets show that FedCon
achieves the best performance with the contractive framework compared with
state-of-the-art baselines under both IID and Non-IID settings. Besides,
ablation studies demonstrate the characteristics of the proposed FedCon
framework.
Related papers
- (FL)$^2$: Overcoming Few Labels in Federated Semi-Supervised Learning [4.803231218533992]
Federated Learning (FL) is a distributed machine learning framework that trains accurate global models while preserving clients' privacy-sensitive data.
Most FL approaches assume that clients possess labeled data, which is often not the case in practice.
We propose $(FL)2$, a robust training method for unlabeled clients using sharpness-aware consistency regularization.
arXiv Detail & Related papers (2024-10-30T17:15:02Z) - FedAnchor: Enhancing Federated Semi-Supervised Learning with Label
Contrastive Loss for Unlabeled Clients [19.3885479917635]
Federated learning (FL) is a distributed learning paradigm that facilitates collaborative training of a shared global model across devices.
We propose FedAnchor, an innovative FSSL method that introduces a unique double-head structure, called anchor head, paired with the classification head trained exclusively on labeled anchor data on the server.
Our approach mitigates the confirmation bias and overfitting issues associated with pseudo-labeling techniques based on high-confidence model prediction samples.
arXiv Detail & Related papers (2024-02-15T18:48:21Z) - Personalized Federated Learning with Attention-based Client Selection [57.71009302168411]
We propose FedACS, a new PFL algorithm with an Attention-based Client Selection mechanism.
FedACS integrates an attention mechanism to enhance collaboration among clients with similar data distributions.
Experiments on CIFAR10 and FMNIST validate FedACS's superiority.
arXiv Detail & Related papers (2023-12-23T03:31:46Z) - FedIL: Federated Incremental Learning from Decentralized Unlabeled Data
with Convergence Analysis [23.70951896315126]
This work considers the server with a small labeled dataset and intends to use unlabeled data in multiple clients for semi-supervised learning.
We propose a new framework with a generalized model, Federated Incremental Learning (FedIL), to address the problem of how to utilize labeled data in the server and unlabeled data in clients separately.
arXiv Detail & Related papers (2023-02-23T07:12:12Z) - FedABC: Targeting Fair Competition in Personalized Federated Learning [76.9646903596757]
Federated learning aims to collaboratively train models without accessing their client's local private data.
We propose a novel and generic PFL framework termed Federated Averaging via Binary Classification, dubbed FedABC.
In particular, we adopt the one-vs-all'' training strategy in each client to alleviate the unfair competition between classes.
arXiv Detail & Related papers (2023-02-15T03:42:59Z) - Federated Zero-Shot Learning for Visual Recognition [55.65879596326147]
We propose a novel Federated Zero-Shot Learning FedZSL framework.
FedZSL learns a central model from the decentralized data residing on edge devices.
The effectiveness and robustness of FedZSL are demonstrated by extensive experiments conducted on three zero-shot benchmark datasets.
arXiv Detail & Related papers (2022-09-05T14:49:34Z) - Robust Deep Semi-Supervised Learning: A Brief Introduction [63.09703308309176]
Semi-supervised learning (SSL) aims to improve learning performance by leveraging unlabeled data when labels are insufficient.
SSL with deep models has proven to be successful on standard benchmark tasks.
However, they are still vulnerable to various robustness threats in real-world applications.
arXiv Detail & Related papers (2022-02-12T04:16:41Z) - FedTriNet: A Pseudo Labeling Method with Three Players for Federated
Semi-supervised Learning [24.720014822365684]
In this paper, we propose a novel federated semi-supervised learning method named FedTriNet.
In particular, we propose to use three networks and a dynamic quality control mechanism to generate high-quality pseudo labels for unlabeled data.
Experimental results on three publicly available datasets show that the proposed FedTriNet outperforms state-of-the-art baselines.
arXiv Detail & Related papers (2021-09-12T21:00:25Z) - Federated Unsupervised Representation Learning [56.715917111878106]
We formulate a new problem in federated learning called Federated Unsupervised Representation Learning (FURL) to learn a common representation model without supervision.
FedCA is composed of two key modules: dictionary module to aggregate the representations of samples from each client and share with all clients for consistency of representation space and alignment module to align the representation of each client on a base model trained on a public data.
arXiv Detail & Related papers (2020-10-18T13:28:30Z) - Federated Semi-Supervised Learning with Inter-Client Consistency &
Disjoint Learning [78.88007892742438]
We study two essential scenarios of Federated Semi-Supervised Learning (FSSL) based on the location of the labeled data.
We propose a novel method to tackle the problems, which we refer to as Federated Matching (FedMatch)
arXiv Detail & Related papers (2020-06-22T09:43:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.