Emulating Full Client Participation: A Long-Term Client Selection Strategy for Federated Learning

• URL: http://arxiv.org/abs/2405.13584v1
• Date: Wed, 22 May 2024 12:27:24 GMT
• Title: Emulating Full Client Participation: A Long-Term Client Selection Strategy for Federated Learning
• Authors: Qingming Li, Juzheng Miao, Puning Zhao, Li Zhou, Shouling Ji, Bowen Zhou, Furui Liu,
• Abstract summary: We propose a novel client selection strategy designed to emulate the performance achieved with full client participation. In a single round, we select clients by minimizing the gradient-space estimation error between the client subset and the full client set. In multi-round selection, we introduce a novel individual fairness constraint, which ensures that clients with similar data distributions have similar frequencies of being selected.
• Score: 48.94952630292219
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Client selection significantly affects the system convergence efficiency and is a crucial problem in federated learning. Existing methods often select clients by evaluating each round individually and overlook the necessity for long-term optimization, resulting in suboptimal performance and potential fairness issues. In this study, we propose a novel client selection strategy designed to emulate the performance achieved with full client participation. In a single round, we select clients by minimizing the gradient-space estimation error between the client subset and the full client set. In multi-round selection, we introduce a novel individual fairness constraint, which ensures that clients with similar data distributions have similar frequencies of being selected. This constraint guides the client selection process from a long-term perspective. We employ Lyapunov optimization and submodular functions to efficiently identify the optimal subset of clients, and provide a theoretical analysis of the convergence ability. Experiments demonstrate that the proposed strategy significantly improves both accuracy and fairness compared to previous methods while also exhibiting efficiency by incurring minimal time overhead.

Related papers

• FedDUAL: A Dual-Strategy with Adaptive Loss and Dynamic Aggregation for Mitigating Data Heterogeneity in Federated Learning [12.307490659840845]
  Federated Learning (FL) combines locally optimized models from various clients into a unified global model. FL encounters significant challenges such as performance degradation, slower convergence, and reduced robustness of the global model. We introduce an innovative dual-strategy approach designed to effectively resolve these issues.
  arXiv  Detail & Related papers  (2024-12-05T18:42:29Z)
• Submodular Maximization Approaches for Equitable Client Selection in Federated Learning [4.167345675621377]
  In a conventional Learning framework, client selection for training typically involves the random sampling of a subset of clients in each iteration. This paper introduces two novel methods, namely SUBTRUNC and UNIONFL, designed to address the limitations of random client selection.
  arXiv  Detail & Related papers  (2024-08-24T22:40:31Z)
• Federated Learning While Providing Model as a Service: Joint Training and Inference Optimization [30.305956110710266]
  Federated learning is beneficial for enabling the training of models across distributed clients. Existing work has overlooked the coexistence of model training and inference under clients' limited resources. This paper focuses on the joint optimization of model training and inference to maximize inference performance at clients.
  arXiv  Detail & Related papers  (2023-12-20T09:27:09Z)
• Greedy Shapley Client Selection for Communication-Efficient Federated Learning [32.38170282930876]
  Standard client selection algorithms for Federated Learning (FL) are often unbiased and involve uniform random sampling of clients. We develop a biased client selection strategy, GreedyFed, that identifies and greedily selects the most contributing clients in each communication round. Compared to various client selection strategies on several real-world datasets, GreedyFed demonstrates fast and stable convergence with high accuracy under timing constraints.
  arXiv  Detail & Related papers  (2023-12-14T16:44:38Z)
• Multi-Criteria Client Selection and Scheduling with Fairness Guarantee for Federated Learning Service [17.986744632466515]
  Federated Learning (FL) enables multiple clients to train machine learning models collaboratively without sharing the raw training data. We propose a multi-criteria client selection and scheduling scheme with a fairness guarantee. Our scheme can improve the model quality especially when data are non-iid.
  arXiv  Detail & Related papers  (2023-12-05T16:56:24Z)
• Provably Personalized and Robust Federated Learning [47.50663360022456]
  We propose simple algorithms which identify clusters of similar clients and train a personalized modelper-cluster. The convergence rates of our algorithmsally match those obtained if we knew the true underlying clustering of the clients and are provably robust in the Byzantine setting.
  arXiv  Detail & Related papers  (2023-06-14T09:37:39Z)
• FilFL: Client Filtering for Optimized Client Participation in Federated Learning [71.46173076298957]
  Federated learning enables clients to collaboratively train a model without exchanging local data. Clients participating in the training process significantly impact the convergence rate, learning efficiency, and model generalization. We propose a novel approach, client filtering, to improve model generalization and optimize client participation and training.
  arXiv  Detail & Related papers  (2023-02-13T18:55:31Z)
• Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction [76.26710990597498]
  We show that the class-imbalance of the grouped data from randomly selected clients can lead to significant performance degradation. Based on our key observation, we design an efficient client sampling mechanism, i.e., Federated Class-balanced Sampling (Fed-CBS) In particular, we propose a measure of class-imbalance and then employ homomorphic encryption to derive this measure in a privacy-preserving way.
  arXiv  Detail & Related papers  (2022-09-30T05:42:56Z)
• Straggler-Resilient Personalized Federated Learning [55.54344312542944]
  Federated learning allows training models from samples distributed across a large network of clients while respecting privacy and communication restrictions. We develop a novel algorithmic procedure with theoretical speedup guarantees that simultaneously handles two of these hurdles. Our method relies on ideas from representation learning theory to find a global common representation using all clients' data and learn a user-specific set of parameters leading to a personalized solution for each client.
  arXiv  Detail & Related papers  (2022-06-05T01:14:46Z)
• DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup [58.894901088797376]
  federated learning has emerged as a promising approach for training a global model using data from multiple organizations without leaking their raw data. We propose a general framework to solve the above two challenges simultaneously. We provide comprehensive theoretical analysis including robustness analysis, convergence analysis, and generalization ability.
  arXiv  Detail & Related papers  (2022-04-16T08:08:29Z)
• On the Convergence of Clustered Federated Learning [57.934295064030636]
  In a federated learning system, the clients, e.g. mobile devices and organization participants, usually have different personal preferences or behavior patterns. This paper proposes a novel weighted client-based clustered FL algorithm to leverage the client's group and each client in a unified optimization framework.
  arXiv  Detail & Related papers  (2022-02-13T02:39:19Z)
• Fair and Consistent Federated Learning [48.19977689926562]
  Federated learning (FL) has gain growing interests for its capability of learning from distributed data sources collectively. We propose an FL framework to jointly consider performance consistency and algorithmic fairness across different local clients.
  arXiv  Detail & Related papers  (2021-08-19T01:56:08Z)
• On Bridging Generic and Personalized Federated Learning [18.989191579101586]
  We propose a novel federated learning framework that explicitly decouples a model's dual duties with two prediction tasks. With this two-loss, two-predictor framework we name Federated Robust Decoupling Fed-RoD, the learned model can simultaneously achieve state-of-the-art generic and personalized performance.
  arXiv  Detail & Related papers  (2021-07-02T00:25:48Z)
• Toward Understanding the Influence of Individual Clients in Federated Learning [52.07734799278535]
  Federated learning allows clients to jointly train a global model without sending their private data to a central server. We defined a new notion called em-Influence, quantify this influence over parameters, and proposed an effective efficient model to estimate this metric.
  arXiv  Detail & Related papers  (2020-12-20T14:34:36Z)
• Stochastic Client Selection for Federated Learning with Volatile Clients [41.591655430723186]
  Federated Learning (FL) is a privacy-preserving machine learning paradigm. In each round of synchronous FL training, only a fraction of available clients are chosen to participate. We propose E3CS, a client selection scheme to solve the problem.
  arXiv  Detail & Related papers  (2020-11-17T16:35:24Z)
• Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies [29.127689561987964]
  Federated learning enables a large number of resource-limited client nodes to cooperatively train a model without data sharing. We show that biasing client selection towards clients with higher local loss achieves faster error convergence. We propose Power-of-Choice, a communication- and computation-efficient client selection framework.
  arXiv  Detail & Related papers  (2020-10-03T01:04:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.