Class-Grouped Normalized Momentum and Faster Hyperparameter Exploration to Tackle Class Imbalance in Federated Learning

• URL: http://arxiv.org/abs/2607.01474v1
• Date: Wed, 01 Jul 2026 21:09:50 GMT
• Title: Class-Grouped Normalized Momentum and Faster Hyperparameter Exploration to Tackle Class Imbalance in Federated Learning
• Abstract summary: Class imbalance poses a critical challenge in federated learning (FL), where underrepresented classes suffer from poor predictive performance.<n>We propose FedCGNM, a client-side in FL that partitions classes into a small number of groups based on minimum within-group variance.<n>To efficiently optimize these rates in small-client regimes, we introduce FedHOO, an X-armed-bandit (XAB) based algorithm.
• Score: 16.216002257704826
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Class imbalance poses a critical challenge in federated learning (FL), where underrepresented classes suffer from poor predictive performance yet cannot be addressed by standard centralized techniques due to privacy and heterogeneity constraints. We propose FedCGNM (Federated Class-Grouped Normalized Momentum), a client-side optimizer in FL that partitions classes into a small number of groups based on minimum within-group variance, maintains a momentum per group, normalizes each group momentum to unit length, and uses the summation of the normalized group momentums as an update direction. This design both equalizes gradient magnitude across majority and minority groups and mitigates the noise inherent in rare-class gradients. We further provide a theoretical convergence analysis explicitly accounting for time-varying resampling-rates. Additionally, to efficiently optimize these rates in small-client regimes, we introduce FedHOO, an X-armed-bandit (XAB) based algorithm that exploits federated parallelism that evaluates many combinations of two candidate rates per client at linear cost. Empirical evaluation on four public long-tailed benchmarks and a proprietary chip-defect dataset demonstrates that FedCGNM consistently outperforms baselines, with FedHOO yielding further gains in small-scale federations.

Related papers

• Data-Free Client Contribution Estimation via Logit Maximization for Federated Learning [26.920835271145492]
  We propose a class-wise contribution estimation and aggregation framework based on logit (CELM)<n>CELM does not require sharing raw data, client metadata, or auxiliary public datasets.<n>We evaluate the approach on representative vision benchmarks under controlled non-IID and pathological label splits.
  arXiv  Detail & Related papers  (2026-05-17T05:49:48Z)
• FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection [5.476264433576713]
  FedSLoP is an efficient Federated Learning algorithm.<n>We show that FedSLoP converges to a first-order stationary point at a rate of $O (1/sqrNT)$.
  arXiv  Detail & Related papers  (2026-04-27T03:47:50Z)
• GRCF: Two-Stage Groupwise Ranking and Calibration Framework for Multimodal Sentiment Analysis [20.77940776708036]
  Pairwise ordinal learning frameworks capture relative order by learning from comparisons.<n>They assign uniform importance to all comparisons, failing to adaptively focus on hard-to-rank samples.<n>We propose a Two-Stage Group-wise Ranking and Framework (GRCF) that adapts the philosophy of Group Relative Policy Optimization.<n>GRCF achieves state-of-the-art performance on core regression benchmarks, while also showing strong generalizability in classification tasks.
  arXiv  Detail & Related papers  (2026-01-14T16:26:44Z)
• NDCG-Consistent Softmax Approximation with Accelerated Convergence [67.10365329542365]
  We propose novel loss formulations that align directly with ranking metrics.<n>We integrate the proposed RG losses with the highly efficient Alternating Least Squares (ALS) optimization method.<n> Empirical evaluations on real-world datasets demonstrate that our approach achieves comparable or superior ranking performance.
  arXiv  Detail & Related papers  (2025-06-11T06:59:17Z)
• Interaction-Aware Gaussian Weighting for Clustered Federated Learning [58.92159838586751]
  Federated Learning (FL) emerged as a decentralized paradigm to train models while preserving privacy.<n>We propose a novel clustered FL method, FedGWC (Federated Gaussian Weighting Clustering), which groups clients based on their data distribution.<n>Our experiments on benchmark datasets show that FedGWC outperforms existing FL algorithms in cluster quality and classification accuracy.
  arXiv  Detail & Related papers  (2025-02-05T16:33:36Z)
• How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalance [64.1656365676171]
  Group imbalance has been a known problem in empirical risk minimization. This paper quantifies the impact of individual groups on the sample complexity, the convergence rate, and the average and group-level testing performance.
  arXiv  Detail & Related papers  (2024-03-12T04:38:05Z)
• Adaptive Learning Rate for Follow-the-Regularized-Leader: Competitive Analysis and Best-of-Both-Worlds [46.30750729936261]
  Follow-The-Regularized-Leader (FTRL) is known as an effective and versatile approach in online learning. We formulate the problem of adjusting FTRL's learning rate as a sequential decision-making problem. We propose update rules for learning rate that achieves an upper bound within a constant factor of this lower bound.
  arXiv  Detail & Related papers  (2024-03-01T18:03:49Z)
• FedABC: Targeting Fair Competition in Personalized Federated Learning [76.9646903596757]
  Federated learning aims to collaboratively train models without accessing their client's local private data. We propose a novel and generic PFL framework termed Federated Averaging via Binary Classification, dubbed FedABC. In particular, we adopt the one-vs-all'' training strategy in each client to alleviate the unfair competition between classes.
  arXiv  Detail & Related papers  (2023-02-15T03:42:59Z)
• Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction [76.26710990597498]
  We show that the class-imbalance of the grouped data from randomly selected clients can lead to significant performance degradation. Based on our key observation, we design an efficient client sampling mechanism, i.e., Federated Class-balanced Sampling (Fed-CBS) In particular, we propose a measure of class-imbalance and then employ homomorphic encryption to derive this measure in a privacy-preserving way.
  arXiv  Detail & Related papers  (2022-09-30T05:42:56Z)
• Improved Multi-label Classification under Temporal Concept Drift: Rethinking Group-Robust Algorithms in a Label-Wise Setting [10.655403615541141]
  In document classification we often deal with hundreds of classes, including very infrequent ones. Class imbalance and drift can sometimes be mitigated by resampling the training data, but what if the target distribution is determined by unknown future events? We evaluate several group-robust optimization algorithms, initially proposed to mitigate group-level disparities. We find that Invariant Risk Minimization and Spectral Decoupling outperform sampling-based approaches to class imbalance and concept drift.
  arXiv  Detail & Related papers  (2022-03-15T13:08:03Z)
• Heterogeneous Federated Learning via Grouped Sequential-to-Parallel Training [60.892342868936865]
  Federated learning (FL) is a rapidly growing privacy-preserving collaborative machine learning paradigm. We propose a data heterogeneous-robust FL approach, FedGSP, to address this challenge. We show that FedGSP improves the accuracy by 3.7% on average compared with seven state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-01-31T03:15:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.