EFFL: Egalitarian Fairness in Federated Learning for Mitigating Matthew
Effect
- URL: http://arxiv.org/abs/2309.16338v1
- Date: Thu, 28 Sep 2023 10:51:12 GMT
- Title: EFFL: Egalitarian Fairness in Federated Learning for Mitigating Matthew
Effect
- Authors: Jiashi Gao, Changwu Huang, Ming Tang, Shin Hwei Tan, Xin Yao, Xuetao
Wei
- Abstract summary: We propose Egalitarian Fairness Federated Learning (EFFL) to mitigate the Matthew effect.
EFFL aims for performance optimality, minimizing the empirical risk loss and the bias for each client.
We show EFFL outperforms other state-of-the-art FL algorithms in achieving a high-performance global model.
- Score: 11.24699174877316
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Recent advances in federated learning (FL) enable collaborative training of
machine learning (ML) models from large-scale and widely dispersed clients
while protecting their privacy. However, when different clients' datasets are
heterogeneous, traditional FL mechanisms produce a global model that does not
adequately represent the poorer clients with limited data resources, resulting
in lower accuracy and higher bias on their local data. According to the Matthew
effect, which describes how the advantaged gain more advantage and the
disadvantaged lose more over time, deploying such a global model in client
applications may worsen the resource disparity among the clients and harm the
principles of social welfare and fairness. To mitigate the Matthew effect, we
propose Egalitarian Fairness Federated Learning (EFFL), where egalitarian
fairness refers to the global model learned from FL has: (1) equal accuracy
among clients; (2) equal decision bias among clients. Besides achieving
egalitarian fairness among the clients, EFFL also aims for performance
optimality, minimizing the empirical risk loss and the bias for each client;
both are essential for any ML model training, whether centralized or
decentralized. We formulate EFFL as a constrained multi-constrained
multi-objectives optimization (MCMOO) problem, with the decision bias and
egalitarian fairness as constraints and the minimization of the empirical risk
losses on all clients as multiple objectives to be optimized. We propose a
gradient-based three-stage algorithm to obtain the Pareto optimal solutions
within the constraint space. Extensive experiments demonstrate that EFFL
outperforms other state-of-the-art FL algorithms in achieving a
high-performance global model with enhanced egalitarian fairness among all
clients.
Related papers
- Multi-dimensional Fair Federated Learning [25.07463977553212]
Federated learning (FL) has emerged as a promising collaborative and secure paradigm for training a model from decentralized data.
Group fairness and client fairness are two dimensions of fairness that are important for FL.
We propose a method, called mFairFL, to achieve group fairness and client fairness simultaneously.
arXiv Detail & Related papers (2023-12-09T11:37:30Z) - Fairness-aware Federated Minimax Optimization with Convergence Guarantee [10.727328530242461]
Federated learning (FL) has garnered considerable attention due to its privacy-preserving feature.
The lack of freedom in managing user data can lead to group fairness issues, where models are biased towards sensitive factors such as race or gender.
This paper proposes a novel algorithm, fair federated averaging with augmented Lagrangian method (FFALM), designed explicitly to address group fairness issues in FL.
arXiv Detail & Related papers (2023-07-10T08:45:58Z) - Confidence-aware Personalized Federated Learning via Variational
Expectation Maximization [34.354154518009956]
We present a novel framework for personalized Federated Learning (PFL)
PFL is a distributed learning scheme to train a shared model across clients.
We present a novel framework for PFL based on hierarchical modeling and variational inference.
arXiv Detail & Related papers (2023-05-21T20:12:27Z) - Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape [59.841889495864386]
In federated learning (FL), a cluster of local clients are chaired under the coordination of a global server.
Clients are prone to overfit into their own optima, which extremely deviates from the global objective.
ttfamily FedSMOO adopts a dynamic regularizer to guarantee the local optima towards the global objective.
Our theoretical analysis indicates that ttfamily FedSMOO achieves fast $mathcalO (1/T)$ convergence rate with low bound generalization.
arXiv Detail & Related papers (2023-05-19T10:47:44Z) - Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients.
FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification.
Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
arXiv Detail & Related papers (2023-05-01T20:04:46Z) - Generalized Federated Learning via Sharpness Aware Minimization [22.294290071999736]
We propose a general, effective algorithm, textttFedSAM, based on Sharpness Aware Minimization (SAM) local, and develop a momentum FL algorithm to bridge local and global models.
Empirically, our proposed algorithms substantially outperform existing FL studies and significantly decrease the learning deviation.
arXiv Detail & Related papers (2022-06-06T13:54:41Z) - Fine-tuning Global Model via Data-Free Knowledge Distillation for
Non-IID Federated Learning [86.59588262014456]
Federated Learning (FL) is an emerging distributed learning paradigm under privacy constraint.
We propose a data-free knowledge distillation method to fine-tune the global model in the server (FedFTG)
Our FedFTG significantly outperforms the state-of-the-art (SOTA) FL algorithms and can serve as a strong plugin for enhancing FedAvg, FedProx, FedDyn, and SCAFFOLD.
arXiv Detail & Related papers (2022-03-17T11:18:17Z) - Fair and Consistent Federated Learning [48.19977689926562]
Federated learning (FL) has gain growing interests for its capability of learning from distributed data sources collectively.
We propose an FL framework to jointly consider performance consistency and algorithmic fairness across different local clients.
arXiv Detail & Related papers (2021-08-19T01:56:08Z) - Blockchain Assisted Decentralized Federated Learning (BLADE-FL):
Performance Analysis and Resource Allocation [119.19061102064497]
We propose a decentralized FL framework by integrating blockchain into FL, namely, blockchain assisted decentralized federated learning (BLADE-FL)
In a round of the proposed BLADE-FL, each client broadcasts its trained model to other clients, competes to generate a block based on the received models, and then aggregates the models from the generated block before its local training of the next round.
We explore the impact of lazy clients on the learning performance of BLADE-FL, and characterize the relationship among the optimal K, the learning parameters, and the proportion of lazy clients.
arXiv Detail & Related papers (2021-01-18T07:19:08Z) - Toward Understanding the Influence of Individual Clients in Federated
Learning [52.07734799278535]
Federated learning allows clients to jointly train a global model without sending their private data to a central server.
We defined a new notion called em-Influence, quantify this influence over parameters, and proposed an effective efficient model to estimate this metric.
arXiv Detail & Related papers (2020-12-20T14:34:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.