FedCAT: Towards Accurate Federated Learning via Device Concatenation

• URL: http://arxiv.org/abs/2202.12751v1
• Date: Wed, 23 Feb 2022 10:08:43 GMT
• Title: FedCAT: Towards Accurate Federated Learning via Device Concatenation
• Authors: Ming Hu, Tian Liu, Zhiwei Ling, Zhihao Yue, Mingsong Chen
• Abstract summary: Federated Learning (FL) enables all the involved devices to train a global model collaboratively without exposing their local data privacy. For non-IID scenarios, the classification accuracy of FL models decreases drastically due to the weight divergence caused by data heterogeneity. We introduce a novel FL approach named Fed-Cat that can achieve high model accuracy based on our proposed device selection strategy and device concatenation-based local training method.
• Score: 4.416919766772866
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As a promising distributed machine learning paradigm, Federated Learning (FL) enables all the involved devices to train a global model collaboratively without exposing their local data privacy. However, for non-IID scenarios, the classification accuracy of FL models decreases drastically due to the weight divergence caused by data heterogeneity. Although various FL variants have been studied to improve model accuracy, most of them still suffer from the problem of non-negligible communication and computation overhead. In this paper, we introduce a novel FL approach named Fed-Cat that can achieve high model accuracy based on our proposed device selection strategy and device concatenation-based local training method. Unlike conventional FL methods that aggregate local models trained on individual devices, FedCat periodically aggregates local models after their traversals through a series of logically concatenated devices, which can effectively alleviate the model weight divergence problem. Comprehensive experimental results on four well-known benchmarks show that our approach can significantly improve the model accuracy of state-of-the-art FL methods without causing extra communication overhead.

Related papers

• Can We Theoretically Quantify the Impacts of Local Updates on the Generalization Performance of Federated Learning? [50.03434441234569]
  Federated Learning (FL) has gained significant popularity due to its effectiveness in training machine learning models across diverse sites without requiring direct data sharing. While various algorithms have shown that FL with local updates is a communication-efficient distributed learning framework, the generalization performance of FL with local updates has received comparatively less attention.
  arXiv  Detail & Related papers  (2024-09-05T19:00:18Z)
• Straggler-resilient Federated Learning: Tackling Computation Heterogeneity with Layer-wise Partial Model Training in Mobile Edge Network [4.1813760301635705]
  We propose Federated Partial Model Training (FedPMT), where devices with smaller computational capabilities work on partial models and contribute to the global model. As such, all devices in FedPMT prioritize the most crucial parts of the global model. Empirical results show that FedPMT significantly outperforms the existing benchmark FedDrop.
  arXiv  Detail & Related papers  (2023-11-16T16:30:04Z)
• Over-the-Air Federated Learning and Optimization [52.5188988624998]
  We focus on Federated learning (FL) via edge-the-air computation (AirComp) We describe the convergence of AirComp-based FedAvg (AirFedAvg) algorithms under both convex and non- convex settings. For different types of local updates that can be transmitted by edge devices (i.e., model, gradient, model difference), we reveal that transmitting in AirFedAvg may cause an aggregation error. In addition, we consider more practical signal processing schemes to improve the communication efficiency and extend the convergence analysis to different forms of model aggregation error caused by these signal processing schemes.
  arXiv  Detail & Related papers  (2023-10-16T05:49:28Z)
• Adaptive Model Pruning and Personalization for Federated Learning over Wireless Networks [72.59891661768177]
  Federated learning (FL) enables distributed learning across edge devices while protecting data privacy. We consider a FL framework with partial model pruning and personalization to overcome these challenges. This framework splits the learning model into a global part with model pruning shared with all devices to learn data representations and a personalized part to be fine-tuned for a specific device.
  arXiv  Detail & Related papers  (2023-09-04T21:10:45Z)
• FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation [16.019513233021435]
  Federated Learning (FL) has attracted increasing attention to deal with data silo problems without compromising user privacy. We present an efficient FL framework named FedCross, which uses a novel multi-to-multi FL training scheme based on our proposed multi-model cross-aggregation approach. We show that FedCross can significantly improve FL accuracy within both IID and non-IID scenarios without causing additional communication overhead.
  arXiv  Detail & Related papers  (2022-10-15T13:12:11Z)
• FedMR: Fedreated Learning via Model Recombination [7.404225808071622]
  Federated Learning (FL) enables global model training across clients without compromising their confidential local data. Existing FL methods rely on Federated Averaging (FedAvg)-based aggregation. This paper proposes a novel and effective FL paradigm named FedMR (Federating Model Recombination)
  arXiv  Detail & Related papers  (2022-08-16T11:30:19Z)
• Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning [86.59588262014456]
  Federated Learning (FL) is an emerging distributed learning paradigm under privacy constraint. We propose a data-free knowledge distillation method to fine-tune the global model in the server (FedFTG) Our FedFTG significantly outperforms the state-of-the-art (SOTA) FL algorithms and can serve as a strong plugin for enhancing FedAvg, FedProx, FedDyn, and SCAFFOLD.
  arXiv  Detail & Related papers  (2022-03-17T11:18:17Z)
• FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorization [16.704006420306353]
  A scalable federated learning framework should address heterogeneous clients equipped with different computation and communication capabilities. This paper proposes FedHM, a novel federated model compression framework that distributes the heterogeneous low-rank models to clients and then aggregates them into a global full-rank model. Our solution enables the training of heterogeneous local models with varying computational complexities and aggregates a single global model.
  arXiv  Detail & Related papers  (2021-11-29T16:11:09Z)
• Over-the-Air Federated Learning from Heterogeneous Data [107.05618009955094]
  Federated learning (FL) is a framework for distributed learning of centralized models. We develop a Convergent OTA FL (COTAF) algorithm which enhances the common local gradient descent (SGD) FL algorithm. We numerically show that the precoding induced by COTAF notably improves the convergence rate and the accuracy of models trained via OTA FL.
  arXiv  Detail & Related papers  (2020-09-27T08:28:25Z)
• Federated Learning With Quantized Global Model Updates [84.55126371346452]
  We study federated learning, which enables mobile devices to utilize their local datasets to train a global model. We introduce a lossy FL (LFL) algorithm, in which both the global model and the local model updates are quantized before being transmitted.
  arXiv  Detail & Related papers  (2020-06-18T16:55:20Z)
• Continual Local Training for Better Initialization of Federated Models [14.289213162030816]
  Federated learning (FL) refers to the learning paradigm that trains machine learning models directly in decentralized systems. The popular FL algorithm emphFederated Averaging (FedAvg) suffers from weight divergence. We propose the local continual training strategy to address this problem.
  arXiv  Detail & Related papers  (2020-05-26T12:27:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.