Expediting In-Network Federated Learning by Voting-Based Consensus Model Compression

• URL: http://arxiv.org/abs/2402.03815v1
• Date: Tue, 6 Feb 2024 09:00:05 GMT
• Title: Expediting In-Network Federated Learning by Voting-Based Consensus Model Compression
• Authors: Xiaoxin Su, Yipeng Zhou, Laizhong Cui and Song Guo
• Abstract summary: We propose Federated Learning in-network Aggregation with Compression (FediAC) algorithm, consisting of two phases: client voting and model aggregating. FediAC consumes much less memory space and communication traffic than existing works because the first phase can guarantee consensus compression across clients. We conduct extensive experiments by using public datasets to demonstrate that FediAC remarkably surpasses the state-of-the-art baselines in terms of model accuracy and communication traffic.
• Score: 28.688895217988925
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, federated learning (FL) has gained momentum because of its capability in preserving data privacy. To conduct model training by FL, multiple clients exchange model updates with a parameter server via Internet. To accelerate the communication speed, it has been explored to deploy a programmable switch (PS) in lieu of the parameter server to coordinate clients. The challenge to deploy the PS in FL lies in its scarce memory space, prohibiting running memory consuming aggregation algorithms on the PS. To overcome this challenge, we propose Federated Learning in-network Aggregation with Compression (FediAC) algorithm, consisting of two phases: client voting and model aggregating. In the former phase, clients report their significant model update indices to the PS to estimate global significant model updates. In the latter phase, clients upload global significant model updates to the PS for aggregation. FediAC consumes much less memory space and communication traffic than existing works because the first phase can guarantee consensus compression across clients. The PS easily aligns model update indices to swiftly complete aggregation in the second phase. Finally, we conduct extensive experiments by using public datasets to demonstrate that FediAC remarkably surpasses the state-of-the-art baselines in terms of model accuracy and communication traffic.

Related papers

• FedAR: Addressing Client Unavailability in Federated Learning with Local Update Approximation and Rectification [8.747592727421596]
  Federated learning (FL) enables clients to collaboratively train machine learning models under the coordination of a server. FedAR can get all clients involved in the global model update to achieve a high-quality global model on the server. FedAR also depicts impressive performance in the presence of a large number of clients with severe client unavailability.
  arXiv  Detail & Related papers  (2024-07-26T21:56:52Z)
• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• Fed-CVLC: Compressing Federated Learning Communications with Variable-Length Codes [54.18186259484828]
  In Federated Learning (FL) paradigm, a parameter server (PS) concurrently communicates with distributed participating clients for model collection, update aggregation, and model distribution over multiple rounds. We show strong evidences that variable-length is beneficial for compression in FL. We present Fed-CVLC (Federated Learning Compression with Variable-Length Codes), which fine-tunes the code length in response to the dynamics of model updates.
  arXiv  Detail & Related papers  (2024-02-06T07:25:21Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• FedCliP: Federated Learning with Client Pruning [3.796320380104124]
  Federated learning (FL) is a newly emerging distributed learning paradigm. One fundamental bottleneck in FL is the heavy communication overheads between the distributed clients and the central server. We propose FedCliP, the first communication efficient FL training framework from a macro perspective.
  arXiv  Detail & Related papers  (2023-01-17T09:15:37Z)
• Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning [28.185096784982544]
  Federated learning (FL) allows multiple clients cooperatively train models without disclosing local data. We propose a heterogeneous-aware FL framework, called FedCG, with adaptive client selection and gradient compression. Experiments on both real-world prototypes and simulations show that FedCG can provide up to 5.3$times$ speedup compared to other methods.
  arXiv  Detail & Related papers  (2022-12-19T14:19:07Z)
• Scalable Collaborative Learning via Representation Sharing [53.047460465980144]
  Federated learning (FL) and Split Learning (SL) are two frameworks that enable collaborative learning while keeping the data private (on device) In FL, each data holder trains a model locally and releases it to a central server for aggregation. In SL, the clients must release individual cut-layer activations (smashed data) to the server and wait for its response (during both inference and back propagation). In this work, we present a novel approach for privacy-preserving machine learning, where the clients collaborate via online knowledge distillation using a contrastive loss.
  arXiv  Detail & Related papers  (2022-11-20T10:49:22Z)
• FedNet2Net: Saving Communication and Computations in Federated Learning with Model Growing [0.0]
  Federated learning (FL) is a recently developed area of machine learning. In this paper, a novel scheme based on the notion of "model growing" is proposed. The proposed approach is tested extensively on three standard benchmarks and is shown to achieve substantial reduction in communication and client computation.
  arXiv  Detail & Related papers  (2022-07-19T21:54:53Z)
• Slashing Communication Traffic in Federated Learning by Transmitting Clustered Model Updates [12.660500431713336]
  Federated Learning (FL) is an emerging decentralized learning framework through which multiple clients can collaboratively train a learning model. heavy communication traffic can be incurred by exchanging model updates via the Internet between clients and the parameter server. In this work, we devise the Model Update Compression by Soft Clustering (MUCSC) algorithm to compress model updates transmitted between clients and the PS.
  arXiv  Detail & Related papers  (2021-05-10T07:15:49Z)
• Timely Communication in Federated Learning [65.1253801733098]
  We consider a global learning framework in which a parameter server (PS) trains a global model by using $n$ clients without actually storing the client data centrally at a cloud server. Under the proposed scheme, at each iteration, the PS waits for $m$ available clients and sends them the current model. We find the average age of information experienced by each client and numerically characterize the age-optimal $m$ and $k$ values for a given $n$.
  arXiv  Detail & Related papers  (2020-12-31T18:52:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.