FedADC: Accelerated Federated Learning with Drift Control

• URL: http://arxiv.org/abs/2012.09102v1
• Date: Wed, 16 Dec 2020 17:49:37 GMT
• Title: FedADC: Accelerated Federated Learning with Drift Control
• Authors: Emre Ozfatura and Kerem Ozfatura and Deniz Gunduz
• Abstract summary: Federated learning (FL) has become de facto framework for collaborative learning among edge devices with privacy concern. Large scale implementation of FL brings new challenges, such as the incorporation of acceleration techniques designed for SGD into the distributed setting, and mitigation of the drift problem due to non-homogeneous distribution of local datasets. We show that it is possible to address both problems using a single strategy without any major alteration to the FL framework, or introducing additional computation and communication load. We propose FedADC, which is an accelerated FL algorithm with drift control.
• Score: 6.746400031322727
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning (FL) has become de facto framework for collaborative learning among edge devices with privacy concern. The core of the FL strategy is the use of stochastic gradient descent (SGD) in a distributed manner. Large scale implementation of FL brings new challenges, such as the incorporation of acceleration techniques designed for SGD into the distributed setting, and mitigation of the drift problem due to non-homogeneous distribution of local datasets. These two problems have been separately studied in the literature; whereas, in this paper, we show that it is possible to address both problems using a single strategy without any major alteration to the FL framework, or introducing additional computation and communication load. To achieve this goal, we propose FedADC, which is an accelerated FL algorithm with drift control. We empirically illustrate the advantages of FedADC.

Related papers

Err

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.