Re-thinking Federated Active Learning based on Inter-class Diversity

• URL: http://arxiv.org/abs/2303.12317v1
• Date: Wed, 22 Mar 2023 05:21:21 GMT
• Title: Re-thinking Federated Active Learning based on Inter-class Diversity
• Authors: SangMook Kim, Sangmin Bae, Hwanjun Song, Se-Young Yun
• Abstract summary: We show that the superiority of two selector models depends on the global and local inter-class diversity. We propose LoGo, a FAL sampling strategy robust to varying local heterogeneity levels and global imbalance ratio. LoGo consistently outperforms six active learning strategies in the total number of 38 experimental settings.
• Score: 16.153683223016973
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although federated learning has made awe-inspiring advances, most studies have assumed that the client's data are fully labeled. However, in a real-world scenario, every client may have a significant amount of unlabeled instances. Among the various approaches to utilizing unlabeled data, a federated active learning framework has emerged as a promising solution. In the decentralized setting, there are two types of available query selector models, namely 'global' and 'local-only' models, but little literature discusses their performance dominance and its causes. In this work, we first demonstrate that the superiority of two selector models depends on the global and local inter-class diversity. Furthermore, we observe that the global and local-only models are the keys to resolving the imbalance of each side. Based on our findings, we propose LoGo, a FAL sampling strategy robust to varying local heterogeneity levels and global imbalance ratio, that integrates both models by two steps of active selection scheme. LoGo consistently outperforms six active learning strategies in the total number of 38 experimental settings.

Related papers

• SFedCA: Credit Assignment-Based Active Client Selection Strategy for Spiking Federated Learning [15.256986486372407]
  Spiking federated learning allows resource-constrained devices to train collaboratively at low power consumption without exchanging local data. Existing spiking federated learning methods employ a random selection approach for client aggregation, assuming unbiased client participation. We propose a credit assignment-based active client selection strategy, the SFedCA, to judiciously aggregate clients that contribute to the global sample distribution balance.
  arXiv  Detail & Related papers  (2024-06-18T01:56:22Z)
• Adaptive Global-Local Representation Learning and Selection for Cross-Domain Facial Expression Recognition [54.334773598942775]
  Domain shift poses a significant challenge in Cross-Domain Facial Expression Recognition (CD-FER) We propose an Adaptive Global-Local Representation Learning and Selection framework.
  arXiv  Detail & Related papers  (2024-01-20T02:21:41Z)
• Federated Skewed Label Learning with Logits Fusion [23.062650578266837]
  Federated learning (FL) aims to collaboratively train a shared model across multiple clients without transmitting their local data. We propose FedBalance, which corrects the optimization bias among local models by calibrating their logits. Our method can gain 13% higher average accuracy compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-11-14T14:37:33Z)
• Rethinking Client Drift in Federated Learning: A Logit Perspective [125.35844582366441]
  Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection. We find that the difference in logits between the local and global models increases as the model is continuously updated. We propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models.
  arXiv  Detail & Related papers  (2023-08-20T04:41:01Z)
• A Global Model Approach to Robust Few-Shot SAR Automatic Target Recognition [6.260916845720537]
  It may not always be possible to collect hundreds of labeled samples per class for training deep learning-based SAR Automatic Target Recognition (ATR) models. This work specifically tackles the few-shot SAR ATR problem, where only a handful of labeled samples may be available to support the task of interest.
  arXiv  Detail & Related papers  (2023-03-20T00:24:05Z)
• Federated Semi-Supervised Learning with Annotation Heterogeneity [57.12560313403097]
  We propose a novel framework called Heterogeneously Annotated Semi-Supervised LEarning (HASSLE) It is a dual-model framework with two models trained separately on labeled and unlabeled data. The dual models can implicitly learn from both types of data across different clients, although each dual model is only trained locally on a single type of data.
  arXiv  Detail & Related papers  (2023-03-04T16:04:49Z)
• Optimizing Server-side Aggregation For Robust Federated Learning via Subspace Training [80.03567604524268]
  Non-IID data distribution across clients and poisoning attacks are two main challenges in real-world federated learning systems. We propose SmartFL, a generic approach that optimize the server-side aggregation process. We provide theoretical analyses of the convergence and generalization capacity for SmartFL.
  arXiv  Detail & Related papers  (2022-11-10T13:20:56Z)
• FedNoiL: A Simple Two-Level Sampling Method for Federated Learning with Noisy Labels [49.47228898303909]
  Federated learning (FL) aims at training a global model on the server side while the training data are collected and located at the local devices. Local training on noisy labels can easily result in overfitting to noisy labels, which is devastating to the global model through aggregation. We develop a simple two-level sampling method "FedNoiL" that selects clients for more robust global aggregation on the server.
  arXiv  Detail & Related papers  (2022-05-20T12:06:39Z)
• Subspace Learning for Personalized Federated Optimization [7.475183117508927]
  We propose a method to address the problem of personalized learning in AI systems. We show that our method achieves consistent gains both in personalized and unseen client evaluation settings.
  arXiv  Detail & Related papers  (2021-09-16T00:03:23Z)
• An Experimental Study of Class Imbalance in Federated Learning [0.8122270502556371]
  We propose two new metrics to define class imbalance -- the global class imbalance degree (MID) and the local difference of class imbalance among clients (WCS) Our results show that a higher MID and a larger WCS degrade more the performance of the global model.
  arXiv  Detail & Related papers  (2021-09-09T08:26:16Z)
• Multi-Center Federated Learning [62.57229809407692]
  This paper proposes a novel multi-center aggregation mechanism for federated learning. It learns multiple global models from the non-IID user data and simultaneously derives the optimal matching between users and centers. Our experimental results on benchmark datasets show that our method outperforms several popular federated learning methods.
  arXiv  Detail & Related papers  (2020-05-03T09:14:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.