Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learning

• URL: http://arxiv.org/abs/2606.10250v1
• Date: Mon, 08 Jun 2026 23:36:29 GMT
• Title: Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learning
• Abstract summary: Class imbalance is a common problem in deep learning that severely degrades performance.<n>We propose FedBB, which consists of two main components: Positive Negative Balanced (PNB) loss function and Client Balanced Reweighting (CBR)<n>Various experiments on X-ray and natural image datasets demonstrate that FedBB outperforms other algorithms in both performance and efficiency.
• Score: 4.791233143264229
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Class imbalance is a common problem in deep learning that severely degrades performance. In federated learning (FL), it is a critical factor contributing to non-identically distributed data (non-IID). Building on several previous attempts, we define and analyze imbalance issues in FL at three levels: inter-case, inter-class, and inter-client. Inter-case imbalance addresses the imbalance in every single class; inter-class imbalance compares the number of data between different classes. Inter-client imbalance represents different skewness of local data between clients. Based on these concepts, we propose FedBB, which consists of two main components: (1) Positive Negative Balanced (PNB) loss function addresses the inter-case and inter-class imbalances in local training, enhancing generalization on highly skewed local client datasets. It optimizes both multi-label and multi-class classifications by assigning higher weights to minority cases or classes. (2) Client Balanced Reweighting (CBR) reweights clients based on inter-client imbalance during model aggregation, giving greater weight to models trained on less skewed datasets. Various experiments on X-ray and natural image datasets demonstrate that FedBB outperforms other algorithms in both performance and efficiency. Additionally, it requires limited statistical information, which is beneficial for privacy protection. Through ablation studies, we proved that PNB loss and CBR independently contribute to performance. As FedBB aims to build a global model that accurately classifies all classes, it can serve as a baseline for the generic and personalized FL.

Related papers

• FedReLa: Imbalanced Federated Learning via Re-Labeling [47.73320550761897]
  Federated learning has emerged as the foremost approach for decentralized model training with privacy preservation.<n>The agnosticism of global class distribution poses significant challenges for data-level methods.<n>We propose FedReLa, a novel data-level approach that tackles the coexistence of data heterogeneity and class imbalance in federated learning.
  arXiv  Detail & Related papers  (2026-06-24T17:13:41Z)
• FedSM: Robust Semantics-Guided Feature Mixup for Bias Reduction in Federated Learning with Long-Tail Data [44.91570926822939]
  Federated Learning (FL) enables collaborative model training across decentralized clients without sharing private data.<n>FL suffers from biased global models due to non-IID and long-tail data distributions.<n>We propose textbfFedSM, a novel client-centric framework that mitigates this bias through semantics-guided feature mixup.
  arXiv  Detail & Related papers  (2025-10-31T07:09:41Z)
• FILM: Framework for Imbalanced Learning Machines based on a new unbiased performance measure and a new ensemble-based technique [37.94431794242543]
  This research addresses the challenges of handling unbalanced datasets for binary classification tasks.<n>Standard evaluation metrics are often biased by the disproportionate representation of the minority class.<n>We propose a novel metric, the Unbiased Integration Coefficients, which exhibits significantly reduced bias.
  arXiv  Detail & Related papers  (2025-03-06T12:15:56Z)
• Hybrid-Regularized Magnitude Pruning for Robust Federated Learning under Covariate Shift [2.298932494750101]
  We show that inconsistencies in client-side training distributions substantially degrade the performance of federated learning models.<n>We propose a novel FL framework using a combination of pruning and regularisation of clients' training to improve the sparsity, redundancy, and robustness of neural connections.
  arXiv  Detail & Related papers  (2024-12-19T16:22:37Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• Addressing Class Variable Imbalance in Federated Semi-supervised Learning [10.542178602467885]
  We propose Federated Semi-supervised Learning for Class Variable Imbalance (FCVI) to solve class variable imbalance. FCVI is used to mitigate the data imbalance due to changes of the number of classes. Our scheme is proved to be significantly better than baseline methods, while maintaining client privacy.
  arXiv  Detail & Related papers  (2023-03-21T12:50:17Z)
• FedABC: Targeting Fair Competition in Personalized Federated Learning [76.9646903596757]
  Federated learning aims to collaboratively train models without accessing their client's local private data. We propose a novel and generic PFL framework termed Federated Averaging via Binary Classification, dubbed FedABC. In particular, we adopt the one-vs-all'' training strategy in each client to alleviate the unfair competition between classes.
  arXiv  Detail & Related papers  (2023-02-15T03:42:59Z)
• Integrating Local Real Data with Global Gradient Prototypes for Classifier Re-Balancing in Federated Long-Tailed Learning [60.41501515192088]
  Federated Learning (FL) has become a popular distributed learning paradigm that involves multiple clients training a global model collaboratively. The data samples usually follow a long-tailed distribution in the real world, and FL on the decentralized and long-tailed data yields a poorly-behaved global model. In this work, we integrate the local real data with the global gradient prototypes to form the local balanced datasets.
  arXiv  Detail & Related papers  (2023-01-25T03:18:10Z)
• Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction [76.26710990597498]
  We show that the class-imbalance of the grouped data from randomly selected clients can lead to significant performance degradation. Based on our key observation, we design an efficient client sampling mechanism, i.e., Federated Class-balanced Sampling (Fed-CBS) In particular, we propose a measure of class-imbalance and then employ homomorphic encryption to derive this measure in a privacy-preserving way.
  arXiv  Detail & Related papers  (2022-09-30T05:42:56Z)
• An Experimental Study of Class Imbalance in Federated Learning [0.8122270502556371]
  We propose two new metrics to define class imbalance -- the global class imbalance degree (MID) and the local difference of class imbalance among clients (WCS) Our results show that a higher MID and a larger WCS degrade more the performance of the global model.
  arXiv  Detail & Related papers  (2021-09-09T08:26:16Z)
• Long-Tailed Recognition Using Class-Balanced Experts [128.73438243408393]
  We propose an ensemble of class-balanced experts that combines the strength of diverse classifiers. Our ensemble of class-balanced experts reaches results close to state-of-the-art and an extended ensemble establishes a new state-of-the-art on two benchmarks for long-tailed recognition.
  arXiv  Detail & Related papers  (2020-04-07T20:57:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.