Loss Gap Parity for Fairness in Heterogeneous Federated Learning

• URL: http://arxiv.org/abs/2603.29818v1
• Date: Tue, 31 Mar 2026 14:43:59 GMT
• Title: Loss Gap Parity for Fairness in Heterogeneous Federated Learning
• Abstract summary: We show that clients prioritize the furthest from their local optimal loss, while maintaining a competitive utility in both convex and non- convex cases.<n>We propose a novel model to minimize disparities in loss gaps across clients.
• Score: 15.28811768542599
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While clients may join federated learning to improve performance on data they rarely observe locally, they often remain self-interested, expecting the global model to perform well on their own data. This motivates an objective that ensures all clients achieve a similar loss gap -the difference in performance between the global model and the best model they could train using only their local data-. To this end, we propose EAGLE, a novel federated learning algorithm that explicitly regularizes the global model to minimize disparities in loss gaps across clients. Our approach is particularly effective in heterogeneous settings, where the optimal local models of the clients may be misaligned. Unlike existing methods that encourage loss parity, potentially degrading performance for many clients, EAGLE targets fairness in relative improvements. We provide theoretical convergence guarantees for EAGLE under non-convex loss functions, and characterize how its iterates perform relative to the standard federated learning objective using a novel heterogeneity measure. Empirically, we demonstrate that EAGLE reduces the disparity in loss gaps among clients by prioritizing those furthest from their local optimal loss, while maintaining competitive utility in both convex and non-convex cases compared to strong baselines.

Related papers

• Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection [53.45696787935487]
  Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes.<n>In real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID.<n>We propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection.
  arXiv  Detail & Related papers  (2026-02-01T05:54:59Z)
• Why Federated Optimization Fails to Achieve Perfect Fitting? A Theoretical Perspective on Client-Side Optima [6.77158446020776]
  This paper provides a theoretical perspective that explains why such degradation occurs.<n>We introduce the assumption that heterogeneous client data lead to distinct local optima.<n>We show that this assumption implies two key consequences: 1) the distance among clients' local optima raises the lower bound of the global objective, making perfect fitting of all client data impossible; and 2) in the final training stage, the global model oscillates within a region instead of converging to a single optimum, limiting its ability to fully fit the data.
  arXiv  Detail & Related papers  (2025-11-01T09:31:35Z)
• Client-Centric Federated Adaptive Optimization [78.30827455292827]
  Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private.<n>We propose Federated-Centric Adaptive Optimization, which is a class of novel federated optimization approaches.
  arXiv  Detail & Related papers  (2025-01-17T04:00:50Z)
• FedDUAL: A Dual-Strategy with Adaptive Loss and Dynamic Aggregation for Mitigating Data Heterogeneity in Federated Learning [12.307490659840845]
  Federated Learning (FL) combines locally optimized models from various clients into a unified global model.<n>FL encounters significant challenges such as performance degradation, slower convergence, and reduced robustness of the global model.<n>We introduce an innovative dual-strategy approach designed to effectively resolve these issues.
  arXiv  Detail & Related papers  (2024-12-05T18:42:29Z)
• Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization [81.32266996009575]
  In federated learning (FL), the multi-step update and data heterogeneity among clients often lead to a loss landscape with sharper minima. We propose FedLESAM, a novel algorithm that locally estimates the direction of global perturbation on client side.
  arXiv  Detail & Related papers  (2024-05-29T08:46:21Z)
• Anti-Matthew FL: Bridging the Performance Gap in Federated Learning to Counteract the Matthew Effect [4.716839088197377]
  Federated learning (FL) facilitates model training across heterogeneous and diverse datasets. In this work, we propose anti-Matthew fairness for the global model at the client level. We show that our proposed anti-Matthew FL outperforms other state-of-the-art FL algorithms in achieving a high-performance global model.
  arXiv  Detail & Related papers  (2023-09-28T10:51:12Z)
• Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape [59.841889495864386]
  In federated learning (FL), a cluster of local clients are chaired under the coordination of a global server. Clients are prone to overfit into their own optima, which extremely deviates from the global objective. ttfamily FedSMOO adopts a dynamic regularizer to guarantee the local optima towards the global objective. Our theoretical analysis indicates that ttfamily FedSMOO achieves fast $mathcalO (1/T)$ convergence rate with low bound generalization.
  arXiv  Detail & Related papers  (2023-05-19T10:47:44Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• Entropy-driven Fair and Effective Federated Learning [26.22014904183881]
  Federated Learning (FL) enables collaborative model training across distributed devices while preserving data privacy.<n>We propose a novel that leverages Theoretical-based aggregation combined with model and gradient alignments to simultaneously optimize and global model performance.
  arXiv  Detail & Related papers  (2023-01-29T10:02:42Z)
• Fair and Consistent Federated Learning [48.19977689926562]
  Federated learning (FL) has gain growing interests for its capability of learning from distributed data sources collectively. We propose an FL framework to jointly consider performance consistency and algorithmic fairness across different local clients.
  arXiv  Detail & Related papers  (2021-08-19T01:56:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.