A Survey on Class Imbalance in Federated Learning
- URL: http://arxiv.org/abs/2303.11673v1
- Date: Tue, 21 Mar 2023 08:34:23 GMT
- Title: A Survey on Class Imbalance in Federated Learning
- Authors: Jing Zhang, Chuanwen Li, Jianzgong Qi, Jiayuan He
- Abstract summary: Federated learning allows multiple client devices in a network to jointly train a machine learning model without direct exposure of clients' data.
It has been found that models trained with federated learning usually have worse performance than their counterparts trained in the standard centralized learning mode.
- Score: 6.632451878730774
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Federated learning, which allows multiple client devices in a network to
jointly train a machine learning model without direct exposure of clients'
data, is an emerging distributed learning technique due to its nature of
privacy preservation. However, it has been found that models trained with
federated learning usually have worse performance than their counterparts
trained in the standard centralized learning mode, especially when the training
data is imbalanced. In the context of federated learning, data imbalance may
occur either locally one one client device, or globally across many devices.
The complexity of different types of data imbalance has posed challenges to the
development of federated learning technique, especially considering the need of
relieving data imbalance issue and preserving data privacy at the same time.
Therefore, in the literature, many attempts have been made to handle class
imbalance in federated learning. In this paper, we present a detailed review of
recent advancements along this line. We first introduce various types of class
imbalance in federated learning, after which we review existing methods for
estimating the extent of class imbalance without the need of knowing the actual
data to preserve data privacy. After that, we discuss existing methods for
handling class imbalance in FL, where the advantages and disadvantages of the
these approaches are discussed. We also summarize common evaluation metrics for
class imbalanced tasks, and point out potential future directions.
Related papers
- On Homomorphic Encryption Based Strategies for Class Imbalance in Federated Learning [4.322339935902437]
Class imbalance in training datasets can lead to bias and poor generalization in machine learning models.
We propose FLICKER, a privacy preserving framework to address issues related to global class imbalance in federated learning.
arXiv Detail & Related papers (2024-10-28T16:35:40Z) - Update Selective Parameters: Federated Machine Unlearning Based on Model Explanation [46.86767774669831]
We propose a more effective and efficient federated unlearning scheme based on the concept of model explanation.
We select the most influential channels within an already-trained model for the data that need to be unlearned.
arXiv Detail & Related papers (2024-06-18T11:43:20Z) - Simplifying Neural Network Training Under Class Imbalance [77.39968702907817]
Real-world datasets are often highly class-imbalanced, which can adversely impact the performance of deep learning models.
The majority of research on training neural networks under class imbalance has focused on specialized loss functions, sampling techniques, or two-stage training procedures.
We demonstrate that simply tuning existing components of standard deep learning pipelines, such as the batch size, data augmentation, and label smoothing, can achieve state-of-the-art performance without any such specialized class imbalance methods.
arXiv Detail & Related papers (2023-12-05T05:52:44Z) - Addressing Class Variable Imbalance in Federated Semi-supervised
Learning [10.542178602467885]
We propose Federated Semi-supervised Learning for Class Variable Imbalance (FCVI) to solve class variable imbalance.
FCVI is used to mitigate the data imbalance due to changes of the number of classes.
Our scheme is proved to be significantly better than baseline methods, while maintaining client privacy.
arXiv Detail & Related papers (2023-03-21T12:50:17Z) - Integrating Local Real Data with Global Gradient Prototypes for
Classifier Re-Balancing in Federated Long-Tailed Learning [60.41501515192088]
Federated Learning (FL) has become a popular distributed learning paradigm that involves multiple clients training a global model collaboratively.
The data samples usually follow a long-tailed distribution in the real world, and FL on the decentralized and long-tailed data yields a poorly-behaved global model.
In this work, we integrate the local real data with the global gradient prototypes to form the local balanced datasets.
arXiv Detail & Related papers (2023-01-25T03:18:10Z) - CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep
Learning [55.733193075728096]
Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance.
Sample re-weighting methods are popularly used to alleviate this data bias issue.
We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
arXiv Detail & Related papers (2022-02-11T13:49:51Z) - Decentralized Federated Learning Preserves Model and Data Privacy [77.454688257702]
We propose a fully decentralized approach, which allows to share knowledge between trained models.
Students are trained on the output of their teachers via synthetically generated input data.
The results show that an untrained student model, trained on the teachers output reaches comparable F1-scores as the teacher.
arXiv Detail & Related papers (2021-02-01T14:38:54Z) - Addressing Class Imbalance in Federated Learning [10.970632986559547]
Federated learning (FL) is a promising approach for training decentralized data located on local client devices.
We propose a monitoring scheme that can infer the composition of training data for each FL round, and design a new loss function -- textbfRatio Loss to mitigate the impact.
arXiv Detail & Related papers (2020-08-14T07:28:08Z) - WAFFLe: Weight Anonymized Factorization for Federated Learning [88.44939168851721]
In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices.
We propose Weight Anonymized Factorization for Federated Learning (WAFFLe), an approach that combines the Indian Buffet Process with a shared dictionary of weight factors for neural networks.
arXiv Detail & Related papers (2020-08-13T04:26:31Z) - Long-Tailed Recognition Using Class-Balanced Experts [128.73438243408393]
We propose an ensemble of class-balanced experts that combines the strength of diverse classifiers.
Our ensemble of class-balanced experts reaches results close to state-of-the-art and an extended ensemble establishes a new state-of-the-art on two benchmarks for long-tailed recognition.
arXiv Detail & Related papers (2020-04-07T20:57:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.