TurboSVM-FL: Boosting Federated Learning through SVM Aggregation for
Lazy Clients
- URL: http://arxiv.org/abs/2401.12012v4
- Date: Sun, 11 Feb 2024 12:09:51 GMT
- Title: TurboSVM-FL: Boosting Federated Learning through SVM Aggregation for
Lazy Clients
- Authors: Mengdi Wang, Anna Bodonhelyi, Efe Bozkir, Enkelejda Kasneci
- Abstract summary: TurboSVM-FL is a novel federated aggregation strategy that poses no additional computation burden on the client side.
We evaluate TurboSVM-FL on multiple datasets including FEMNIST, CelebA, and Shakespeare.
- Score: 44.44776028287441
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Federated learning is a distributed collaborative machine learning paradigm
that has gained strong momentum in recent years. In federated learning, a
central server periodically coordinates models with clients and aggregates the
models trained locally by clients without necessitating access to local data.
Despite its potential, the implementation of federated learning continues to
encounter several challenges, predominantly the slow convergence that is
largely due to data heterogeneity. The slow convergence becomes particularly
problematic in cross-device federated learning scenarios where clients may be
strongly limited by computing power and storage space, and hence counteracting
methods that induce additional computation or memory cost on the client side
such as auxiliary objective terms and larger training iterations can be
impractical. In this paper, we propose a novel federated aggregation strategy,
TurboSVM-FL, that poses no additional computation burden on the client side and
can significantly accelerate convergence for federated classification task,
especially when clients are "lazy" and train their models solely for few epochs
for next global aggregation. TurboSVM-FL extensively utilizes support vector
machine to conduct selective aggregation and max-margin spread-out
regularization on class embeddings. We evaluate TurboSVM-FL on multiple
datasets including FEMNIST, CelebA, and Shakespeare using user-independent
validation with non-iid data distribution. Our results show that TurboSVM-FL
can significantly outperform existing popular algorithms on convergence rate
and reduce communication rounds while delivering better test metrics including
accuracy, F1 score, and MCC.
Related papers
- FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clustering [26.478852701376294]
Federated learning (FL) is an emerging distributed machine learning paradigm.
One of the major challenges in FL is the presence of uneven data distributions across client devices.
We propose em FedClust, a novel approach for CFL that leverages the correlation between local model weights and the data distribution of clients.
arXiv Detail & Related papers (2024-07-09T02:47:16Z) - FedClust: Optimizing Federated Learning on Non-IID Data through
Weight-Driven Client Clustering [28.057411252785176]
Federated learning (FL) is an emerging distributed machine learning paradigm enabling collaborative model training on decentralized devices without exposing their local data.
This paper proposes FedClust, a novel CFL approach leveraging correlations between local model weights and client data distributions.
arXiv Detail & Related papers (2024-03-07T01:50:36Z) - Achieving Linear Speedup in Asynchronous Federated Learning with
Heterogeneous Clients [30.135431295658343]
Federated learning (FL) aims to learn a common global model without exchanging or transferring the data that are stored locally at different clients.
In this paper, we propose an efficient federated learning (AFL) framework called DeFedAvg.
DeFedAvg is the first AFL algorithm that achieves the desirable linear speedup property, which indicates its high scalability.
arXiv Detail & Related papers (2024-02-17T05:22:46Z) - FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup
for Non-IID Data [54.81695390763957]
Federated learning is an emerging distributed machine learning method.
We propose a heterogeneous local variant of AMSGrad, named FedLALR, in which each client adjusts its learning rate.
We show that our client-specified auto-tuned learning rate scheduling can converge and achieve linear speedup with respect to the number of clients.
arXiv Detail & Related papers (2023-09-18T12:35:05Z) - Effectively Heterogeneous Federated Learning: A Pairing and Split
Learning Based Approach [16.093068118849246]
This paper presents a novel split federated learning (SFL) framework that pairs clients with different computational resources.
A greedy algorithm is proposed by reconstructing the optimization of training latency as a graph edge selection problem.
Simulation results show the proposed method can significantly improve the FL training speed and achieve high performance.
arXiv Detail & Related papers (2023-08-26T11:10:54Z) - Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training.
In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework.
Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
arXiv Detail & Related papers (2023-08-11T09:58:47Z) - Optimizing Server-side Aggregation For Robust Federated Learning via
Subspace Training [80.03567604524268]
Non-IID data distribution across clients and poisoning attacks are two main challenges in real-world federated learning systems.
We propose SmartFL, a generic approach that optimize the server-side aggregation process.
We provide theoretical analyses of the convergence and generalization capacity for SmartFL.
arXiv Detail & Related papers (2022-11-10T13:20:56Z) - Federated Stochastic Gradient Descent Begets Self-Induced Momentum [151.4322255230084]
Federated learning (FL) is an emerging machine learning method that can be applied in mobile edge systems.
We show that running to the gradient descent (SGD) in such a setting can be viewed as adding a momentum-like term to the global aggregation process.
arXiv Detail & Related papers (2022-02-17T02:01:37Z) - Straggler-Resilient Federated Learning: Leveraging the Interplay Between
Statistical Accuracy and System Heterogeneity [57.275753974812666]
Federated learning involves learning from data samples distributed across a network of clients while the data remains local.
In this paper, we propose a novel straggler-resilient federated learning method that incorporates statistical characteristics of the clients' data to adaptively select the clients in order to speed up the learning procedure.
arXiv Detail & Related papers (2020-12-28T19:21:14Z) - Coded Computing for Federated Learning at the Edge [3.385874614913973]
Federated Learning (FL) enables training a global model from data generated locally at the client nodes, without moving client data to a centralized server.
Recent work proposes to mitigate stragglers and speed up training for linear regression tasks by assigning redundant computations at the MEC server.
We develop CodedFedL that addresses the difficult task of extending CFL to distributed non-linear regression and classification problems with multioutput labels.
arXiv Detail & Related papers (2020-07-07T08:20:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.