Towards Federated Learning With Byzantine-Robust Client Weighting

• URL: http://arxiv.org/abs/2004.04986v2
• Date: Tue, 18 May 2021 08:10:10 GMT
• Title: Towards Federated Learning With Byzantine-Robust Client Weighting
• Authors: Amit Portnoy, Yoav Tirosh, and Danny Hendler
• Abstract summary: We propose a practical weight-truncation-based preprocessing method for Federated Learning. We show that our method can strike a good balance between model quality and Byzantine robustness.
• Score: 2.294014185517203
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Federated Learning (FL) is a distributed machine learning paradigm where data is distributed among clients who collaboratively train a model in a computation process coordinated by a central server. By assigning a weight to each client based on the proportion of data instances it possesses, the rate of convergence to an accurate joint model can be greatly accelerated. Some previous works studied FL in a Byzantine setting, in which a fraction of the clients may send arbitrary or even malicious information regarding their model. However, these works either ignore the issue of data unbalancedness altogether or assume that client weights are apriori known to the server, whereas, in practice, it is likely that weights will be reported to the server by the clients themselves and therefore cannot be relied upon. We address this issue for the first time by proposing a practical weight-truncation-based preprocessing method and demonstrating empirically that it is able to strike a good balance between model quality and Byzantine robustness. We also establish analytically that our method can be applied to a randomly selected sample of client weights.

Related papers

• Personalized Federated Learning with Mixture of Models for Adaptive Prediction and Model Fine-Tuning [22.705411388403036]
  This paper develops a novel personalized federated learning algorithm. Each client constructs a personalized model by combining a locally fine-tuned model with multiple federated models. Theoretical analysis and experiments on real datasets corroborate the effectiveness of this approach.
  arXiv  Detail & Related papers  (2024-10-28T21:20:51Z)
• Cohort Squeeze: Beyond a Single Communication Round per Cohort in Cross-Device Federated Learning [51.560590617691005]
  We investigate whether it is possible to squeeze more juice" out of each cohort than what is possible in a single communication round. Our approach leads to up to 74% reduction in the total communication cost needed to train a FL model in the cross-device setting.
  arXiv  Detail & Related papers  (2024-06-03T08:48:49Z)
• Stochastic Approximation Approach to Federated Machine Learning [0.0]
  This paper examines Federated learning (FL) in a Approximation (SA) framework. FL is a collaborative way to train neural network models across various participants or clients. It is observed that the proposed algorithm is robust and gives more reliable estimates of the weights.
  arXiv  Detail & Related papers  (2024-02-20T12:00:25Z)
• FedSampling: A Better Sampling Strategy for Federated Learning [81.85411484302952]
  Federated learning (FL) is an important technique for learning models from decentralized data in a privacy-preserving way. Existing FL methods usually uniformly sample clients for local model learning in each round. We propose a novel data uniform sampling strategy for federated learning (FedSampling)
  arXiv  Detail & Related papers  (2023-06-25T13:38:51Z)
• FedDWA: Personalized Federated Learning with Dynamic Weight Adjustment [20.72576355616359]
  We propose a new PFL algorithm called emphFedDWA (Federated Learning with Dynamic Weight Adjustment) to address the problem. FedDWA computes personalized aggregation weights based on collected models from clients. We conduct extensive experiments using five real datasets and the results demonstrate that FedDWA can significantly reduce the communication traffic and achieve much higher model accuracy than the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-05-10T13:12:07Z)
• FedCliP: Federated Learning with Client Pruning [3.796320380104124]
  Federated learning (FL) is a newly emerging distributed learning paradigm. One fundamental bottleneck in FL is the heavy communication overheads between the distributed clients and the central server. We propose FedCliP, the first communication efficient FL training framework from a macro perspective.
  arXiv  Detail & Related papers  (2023-01-17T09:15:37Z)
• Deep Unfolding-based Weighted Averaging for Federated Learning in Heterogeneous Environments [11.023081396326507]
  Federated learning is a collaborative model training method that iterates model updates by multiple clients and aggregation of the updates by a central server. To adjust the aggregation weights, this paper employs deep unfolding, which is known as the parameter tuning method. The proposed method can handle large-scale learning models with the aid of pretrained models such as it can perform practical real-world tasks.
  arXiv  Detail & Related papers  (2022-12-23T08:20:37Z)
• Optimizing Server-side Aggregation For Robust Federated Learning via Subspace Training [80.03567604524268]
  Non-IID data distribution across clients and poisoning attacks are two main challenges in real-world federated learning systems. We propose SmartFL, a generic approach that optimize the server-side aggregation process. We provide theoretical analyses of the convergence and generalization capacity for SmartFL.
  arXiv  Detail & Related papers  (2022-11-10T13:20:56Z)
• A Bayesian Federated Learning Framework with Online Laplace Approximation [144.7345013348257]
  Federated learning allows multiple clients to collaboratively learn a globally shared model. We propose a novel FL framework that uses online Laplace approximation to approximate posteriors on both the client and server side. We achieve state-of-the-art results on several benchmarks, clearly demonstrating the advantages of the proposed method.
  arXiv  Detail & Related papers  (2021-02-03T08:36:58Z)
• Toward Understanding the Influence of Individual Clients in Federated Learning [52.07734799278535]
  Federated learning allows clients to jointly train a global model without sending their private data to a central server. We defined a new notion called em-Influence, quantify this influence over parameters, and proposed an effective efficient model to estimate this metric.
  arXiv  Detail & Related papers  (2020-12-20T14:34:36Z)
• Personalized Federated Learning with First Order Model Optimization [76.81546598985159]
  We propose an alternative to federated learning, where each client federates with other relevant clients to obtain a stronger model per client-specific objectives. We do not assume knowledge of underlying data distributions or client similarities, and allow each client to optimize for arbitrary target distributions of interest. Our method outperforms existing alternatives, while also enabling new features for personalized FL such as transfer outside of local data distributions.
  arXiv  Detail & Related papers  (2020-12-15T19:30:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.