Coded Matrix Computations for D2D-enabled Linearized Federated Learning

• URL: http://arxiv.org/abs/2302.12305v1
• Date: Thu, 23 Feb 2023 20:01:46 GMT
• Title: Coded Matrix Computations for D2D-enabled Linearized Federated Learning
• Authors: Anindya Bijoy Das, Aditya Ramamoorthy, David J. Love, Christopher G. Brinton
• Abstract summary: Federated learning (FL) is a popular technique for training a global model on data distributed across client devices. Recent work has proposed to address this through device-to-device (D2D) offloading, which introduces privacy concerns. We propose a novel straggler-optimal approach for coded matrix computations which can significantly reduce the communication delay and privacy issues.
• Score: 26.086036387834866
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning (FL) is a popular technique for training a global model on data distributed across client devices. Like other distributed training techniques, FL is susceptible to straggler (slower or failed) clients. Recent work has proposed to address this through device-to-device (D2D) offloading, which introduces privacy concerns. In this paper, we propose a novel straggler-optimal approach for coded matrix computations which can significantly reduce the communication delay and privacy issues introduced from D2D data transmissions in FL. Moreover, our proposed approach leads to a considerable improvement of the local computation speed when the generated data matrix is sparse. Numerical evaluations confirm the superiority of our proposed method over baseline approaches.

Related papers

• Device Sampling and Resource Optimization for Federated Learning in Cooperative Edge Networks [17.637761046608]
  Federated learning (FedL) distributes machine learning (ML) across worker devices by having them train local models that are periodically aggregated by a server. FedL ignores two important characteristics of contemporary wireless networks: (i) the network may contain heterogeneous communication/computation resources, and (ii) there may be significant overlaps in devices' local data distributions. We develop a novel optimization methodology that jointly accounts for these factors via intelligent device sampling complemented by device-to-device (D2D) offloading.
  arXiv  Detail & Related papers  (2023-11-07T21:17:59Z)
• Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity [72.85248553787538]
  This paper focuses on performance analysis and optimization for wireless FL, considering data heterogeneity, combined with wireless resource allocation. We formulate the loss function minimization problem, under constraints on long-term energy consumption and latency, and jointly optimize client scheduling, resource allocation, and the number of local training epochs (CRE) Experiments on real-world datasets demonstrate that the proposed algorithm outperforms other benchmarks in terms of the learning accuracy and energy consumption.
  arXiv  Detail & Related papers  (2023-08-04T04:18:01Z)
• Gradient Sparsification for Efficient Wireless Federated Learning with Differential Privacy [25.763777765222358]
  Federated learning (FL) enables distributed clients to collaboratively train a machine learning model without sharing raw data with each other. As the model size grows, the training latency due to limited transmission bandwidth and private information degrades while using differential privacy (DP) protection. We propose sparsification empowered FL framework wireless channels, in over to improve training efficiency without sacrificing convergence performance.
  arXiv  Detail & Related papers  (2023-04-09T05:21:15Z)
• sqSGD: Locally Private and Communication Efficient Federated Learning [14.60645909629309]
  Federated learning (FL) is a technique that trains machine learning models from decentralized data sources. We develop a gradient-based learning algorithm called sqSGD that addresses communication efficiency and high-dimensional compatibility. Experiment results show sqSGD successfully learns large models like LeNet and ResNet with local privacy constraints.
  arXiv  Detail & Related papers  (2022-06-21T17:45:35Z)
• Over-the-Air Federated Learning via Second-Order Optimization [37.594140209854906]
  Federated learning (FL) could result in task-oriented data traffic flows over wireless networks with limited radio resources. We propose a novel over-the-air second-order federated optimization algorithm to simultaneously reduce the communication rounds and enable low-latency global model aggregation.
  arXiv  Detail & Related papers  (2022-03-29T12:39:23Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• Dynamic Network-Assisted D2D-Aided Coded Distributed Learning [59.29409589861241]
  We propose a novel device-to-device (D2D)-aided coded federated learning method (D2D-CFL) for load balancing across devices. We derive an optimal compression rate for achieving minimum processing time and establish its connection with the convergence time. Our proposed method is beneficial for real-time collaborative applications, where the users continuously generate training data.
  arXiv  Detail & Related papers  (2021-11-26T18:44:59Z)
• Learning from Images: Proactive Caching with Parallel Convolutional Neural Networks [94.85780721466816]
  A novel framework for proactive caching is proposed in this paper. It combines model-based optimization with data-driven techniques by transforming an optimization problem into a grayscale image. Numerical results show that the proposed scheme can reduce 71.6% computation time with only 0.8% additional performance cost.
  arXiv  Detail & Related papers  (2021-08-15T21:32:47Z)
• LoRD-Net: Unfolded Deep Detection Network with Low-Resolution Receivers [104.01415343139901]
  We propose a deep detector entitled LoRD-Net for recovering information symbols from one-bit measurements. LoRD-Net has a task-based architecture dedicated to recovering the underlying signal of interest. We evaluate the proposed receiver architecture for one-bit signal recovery in wireless communications.
  arXiv  Detail & Related papers  (2021-02-05T04:26:05Z)
• A Compressive Sensing Approach for Federated Learning over Massive MIMO Communication Systems [82.2513703281725]
  Federated learning is a privacy-preserving approach to train a global model at a central server by collaborating with wireless devices. We present a compressive sensing approach for federated learning over massive multiple-input multiple-output communication systems.
  arXiv  Detail & Related papers  (2020-03-18T05:56:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.