Decentralized Learning over Wireless Networks with Broadcast-Based Subgraph Sampling

• URL: http://arxiv.org/abs/2310.16106v1
• Date: Tue, 24 Oct 2023 18:15:52 GMT
• Title: Decentralized Learning over Wireless Networks with Broadcast-Based Subgraph Sampling
• Authors: Daniel P\'erez Herrera, Zheng Chen and Erik G. Larsson
• Abstract summary: This work centers on the communication aspects of decentralized learning over wireless networks, using consensus-based decentralized descent (D-SGD) Considering the actual communication cost or delay caused by in-network information exchange in an iterative process, our goal is to achieve fast convergence of the algorithm measured by improvement per transmission slot. We propose BASS, an efficient communication framework for D-SGD over wireless networks with broadcast transmission and probabilistic subgraph sampling.
• Score: 36.99249604183772
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work centers on the communication aspects of decentralized learning over wireless networks, using consensus-based decentralized stochastic gradient descent (D-SGD). Considering the actual communication cost or delay caused by in-network information exchange in an iterative process, our goal is to achieve fast convergence of the algorithm measured by improvement per transmission slot. We propose BASS, an efficient communication framework for D-SGD over wireless networks with broadcast transmission and probabilistic subgraph sampling. In each iteration, we activate multiple subsets of non-interfering nodes to broadcast model updates to their neighbors. These subsets are randomly activated over time, with probabilities reflecting their importance in network connectivity and subject to a communication cost constraint (e.g., the average number of transmission slots per iteration). During the consensus update step, only bi-directional links are effectively preserved to maintain communication symmetry. In comparison to existing link-based scheduling methods, the inherent broadcasting nature of wireless channels offers intrinsic advantages in speeding up convergence of decentralized learning by creating more communicated links with the same number of transmission slots.

Related papers

• Collaborative Ground-Space Communications via Evolutionary Multi-objective Deep Reinforcement Learning [113.48727062141764]
  We propose a distributed collaborative beamforming (DCB)-based uplink communication paradigm for enabling ground-space direct communications. DCB treats the terminals that are unable to establish efficient direct connections with the low Earth orbit (LEO) satellites as distributed antennas. We propose an evolutionary multi-objective deep reinforcement learning algorithm to obtain the desirable policies.
  arXiv  Detail & Related papers  (2024-04-11T03:13:02Z)
• Faster Convergence with Less Communication: Broadcast-Based Subgraph Sampling for Decentralized Learning over Wireless Networks [32.914407967052114]
  $texttBASS$ is a broadcast-based subgraph sampling method designed to accelerate the convergence of D-SGD. We show that $texttBASS$ enables faster convergence with fewer transmission slots compared to existing link-based scheduling methods.
  arXiv  Detail & Related papers  (2024-01-24T20:00:23Z)
• Communication-Efficient Framework for Distributed Image Semantic Wireless Transmission [68.69108124451263]
  Federated learning-based semantic communication (FLSC) framework for multi-task distributed image transmission with IoT devices. Each link is composed of a hierarchical vision transformer (HVT)-based extractor and a task-adaptive translator. Channel state information-based multiple-input multiple-output transmission module designed to combat channel fading and noise.
  arXiv  Detail & Related papers  (2023-08-07T16:32:14Z)
• Decentralized Learning over Wireless Networks: The Effect of Broadcast with Random Access [56.91063444859008]
  We investigate the impact of broadcast transmission and probabilistic random access policy on the convergence performance of D-SGD. Our results demonstrate that optimizing the access probability to maximize the expected number of successful links is a highly effective strategy for accelerating the system convergence.
  arXiv  Detail & Related papers  (2023-05-12T10:32:26Z)
• Non-Coherent Over-the-Air Decentralized Gradient Descent [0.0]
  Implementing Decentralized Gradient Descent in wireless systems is challenging due to noise, fading, and limited bandwidth. This paper introduces a scalable DGD algorithm that eliminates the need for scheduling, topology information, or CSI.
  arXiv  Detail & Related papers  (2022-11-19T19:15:34Z)
• Asynchronous Decentralized Learning over Unreliable Wireless Networks [4.630093015127539]
  Decentralized learning enables edge users to collaboratively train models by exchanging information via device-to-device communication. We propose an asynchronous decentralized gradient descent (DSGD) algorithm, which is robust to the inherent and communication failures occurring at the wireless network edge. Experimental results corroborate our analysis, demonstrating the benefits of asynchronicity and outdated gradient information reuse in decentralized learning over unreliable wireless networks.
  arXiv  Detail & Related papers  (2022-02-02T11:00:49Z)
• End-to-End Learning for Uplink MU-SIMO Joint Transmitter and Non-Coherent Receiver Design in Fading Channels [11.182920270301304]
  A novel end-to-end learning approach, namely JTRD-Net, is proposed for uplink multiuser single-input multiple-output (MU-SIMO) joint transmitter and non-coherent receiver design (JTRD) in fading channels. The transmitter side is modeled as a group of parallel linear layers, which are responsible for multiuser waveform design. The non-coherent receiver is formed by a deep feed-forward neural network (DFNN) so as to provide multiuser detection (MUD) capabilities.
  arXiv  Detail & Related papers  (2021-05-04T02:47:59Z)
• Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications [55.65768284748698]
  Machine learning (ML) is a promising enabler for the fifth generation (5G) communication systems and beyond. This article aims to provide a holistic overview of relevant communication and ML principles, and thereby present communication-efficient and distributed learning frameworks with selected use cases.
  arXiv  Detail & Related papers  (2020-08-06T12:37:14Z)
• Decentralized Learning for Channel Allocation in IoT Networks over Unlicensed Bandwidth as a Contextual Multi-player Multi-armed Bandit Game [134.88020946767404]
  We study a decentralized channel allocation problem in an ad-hoc Internet of Things network underlaying on the spectrum licensed to a primary cellular network. Our study maps this problem into a contextual multi-player, multi-armed bandit game, and proposes a purely decentralized, three-stage policy learning algorithm through trial-and-error.
  arXiv  Detail & Related papers  (2020-03-30T10:05:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.