SemiFL: Communication Efficient Semi-Supervised Federated Learning with Unlabeled Clients

• URL: http://arxiv.org/abs/2106.01432v1
• Date: Wed, 2 Jun 2021 19:22:26 GMT
• Title: SemiFL: Communication Efficient Semi-Supervised Federated Learning with Unlabeled Clients
• Authors: Enmao Diao, Jie Ding, Vahid Tarokh
• Abstract summary: We propose a new Federated Learning framework referred to as SemiFL. In SemiFL, clients have completely unlabeled data, while the server has a small amount of labeled data. We demonstrate various efficient strategies of SemiFL that enhance learning performance.
• Score: 34.24028216079336
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated Learning allows training machine learning models by using the computation and private data resources of a large number of distributed clients such as smartphones and IoT devices. Most existing works on Federated Learning (FL) assume the clients have ground-truth labels. However, in many practical scenarios, clients may be unable to label task-specific data, e.g., due to lack of expertise. In this work, we consider a server that hosts a labeled dataset, and wishes to leverage clients with unlabeled data for supervised learning. We propose a new Federated Learning framework referred to as SemiFL in order to address the problem of Semi-Supervised Federated Learning (SSFL). In SemiFL, clients have completely unlabeled data, while the server has a small amount of labeled data. SemiFL is communication efficient since it separates the training of server-side supervised data and client-side unsupervised data. We demonstrate various efficient strategies of SemiFL that enhance learning performance. Extensive empirical evaluations demonstrate that our communication efficient method can significantly improve the performance of a labeled server with unlabeled clients. Moreover, we demonstrate that SemiFL can outperform many existing FL results trained with fully supervised data, and perform competitively with the state-of-the-art centralized Semi-Supervised Learning (SSL) methods. For instance, in standard communication efficient scenarios, our method can perform 93% accuracy on the CIFAR10 dataset with only 4000 labeled samples at the server. Such accuracy is only 2% away from the result trained from 50000 fully labeled data, and it improves about 30% upon existing SSFL methods in the communication efficient setting.

Related papers

• (FL)$^2$: Overcoming Few Labels in Federated Semi-Supervised Learning [4.803231218533992]
  Federated Learning (FL) is a distributed machine learning framework that trains accurate global models while preserving clients' privacy-sensitive data. Most FL approaches assume that clients possess labeled data, which is often not the case in practice. We propose $(FL)2$, a robust training method for unlabeled clients using sharpness-aware consistency regularization.
  arXiv  Detail & Related papers  (2024-10-30T17:15:02Z)
• Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach [67.27031215756121]
  Federated learning (FL) is a machine learning paradigm that targets model training without gathering the local data over various data sources. Standard FL, which employs a single server, can only support a limited number of users, leading to degraded learning capability. In this work, we consider a multi-server FL framework, referred to as emphConfederated Learning (CFL) in order to accommodate a larger number of users.
  arXiv  Detail & Related papers  (2024-02-28T03:27:10Z)
• FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data [54.81695390763957]
  Federated learning is an emerging distributed machine learning method. We propose a heterogeneous local variant of AMSGrad, named FedLALR, in which each client adjusts its learning rate. We show that our client-specified auto-tuned learning rate scheduling can converge and achieve linear speedup with respect to the number of clients.
  arXiv  Detail & Related papers  (2023-09-18T12:35:05Z)
• SemiSFL: Split Federated Learning on Unlabeled and Non-IID Data [34.49090830845118]
  Federated Learning (FL) has emerged to allow multiple clients to collaboratively train machine learning models on their private data at the network edge. We propose a novel Semi-supervised SFL system, termed SemiSFL, which incorporates clustering regularization to perform SFL with unlabeled and non-IID client data. Our system provides a 3.8x speed-up in training time, reduces the communication cost by about 70.3% while reaching the target accuracy, and achieves up to 5.8% improvement in accuracy under non-IID scenarios.
  arXiv  Detail & Related papers  (2023-07-29T02:35:37Z)
• FedSampling: A Better Sampling Strategy for Federated Learning [81.85411484302952]
  Federated learning (FL) is an important technique for learning models from decentralized data in a privacy-preserving way. Existing FL methods usually uniformly sample clients for local model learning in each round. We propose a novel data uniform sampling strategy for federated learning (FedSampling)
  arXiv  Detail & Related papers  (2023-06-25T13:38:51Z)
• Knowledge-Enhanced Semi-Supervised Federated Learning for Aggregating Heterogeneous Lightweight Clients in IoT [34.128674870180596]
  Federated learning (FL) enables multiple clients to train models collaboratively without sharing local data. We propose pFedKnow, which generates lightweight personalized client models via neural network pruning techniques to reduce communication cost. Experiment results on both image and text datasets show that the proposed pFedKnow outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-03-05T13:19:10Z)
• Scalable Collaborative Learning via Representation Sharing [53.047460465980144]
  Federated learning (FL) and Split Learning (SL) are two frameworks that enable collaborative learning while keeping the data private (on device) In FL, each data holder trains a model locally and releases it to a central server for aggregation. In SL, the clients must release individual cut-layer activations (smashed data) to the server and wait for its response (during both inference and back propagation). In this work, we present a novel approach for privacy-preserving machine learning, where the clients collaborate via online knowledge distillation using a contrastive loss.
  arXiv  Detail & Related papers  (2022-11-20T10:49:22Z)
• Optimizing Server-side Aggregation For Robust Federated Learning via Subspace Training [80.03567604524268]
  Non-IID data distribution across clients and poisoning attacks are two main challenges in real-world federated learning systems. We propose SmartFL, a generic approach that optimize the server-side aggregation process. We provide theoretical analyses of the convergence and generalization capacity for SmartFL.
  arXiv  Detail & Related papers  (2022-11-10T13:20:56Z)
• Distributed Unsupervised Visual Representation Learning with Fused Features [13.935997509072669]
  Federated learning (FL) enables distributed clients to learn a shared model for prediction while keeping the training data local on each client. We propose a federated contrastive learning framework consisting of two approaches: feature fusion and neighborhood matching. It outperforms other methods by 11% on IID data and matches the performance of centralized learning.
  arXiv  Detail & Related papers  (2021-11-21T08:36:31Z)
• FedSEAL: Semi-Supervised Federated Learning with Self-Ensemble Learning and Negative Learning [7.771967424619346]
  Federated learning (FL) is a popular decentralized and privacy-preserving machine learning (FL) framework. In this paper, we propose a new FL algorithm, called FedSEAL, to solve this Semi-Supervised Federated Learning (SSFL) problem. Our algorithm utilizes self-ensemble learning and complementary negative learning to enhance both the accuracy and the efficiency of clients' unsupervised learning on unlabeled data.
  arXiv  Detail & Related papers  (2021-10-15T03:03:23Z)
• Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning [78.88007892742438]
  We study two essential scenarios of Federated Semi-Supervised Learning (FSSL) based on the location of the labeled data. We propose a novel method to tackle the problems, which we refer to as Federated Matching (FedMatch)
  arXiv  Detail & Related papers  (2020-06-22T09:43:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.