Bayesian Over-the-Air FedAvg via Channel Driven Stochastic Gradient Langevin Dynamics

• URL: http://arxiv.org/abs/2305.04152v2
• Date: Tue, 9 May 2023 13:30:12 GMT
• Title: Bayesian Over-the-Air FedAvg via Channel Driven Stochastic Gradient Langevin Dynamics
• Authors: Boning Zhang, Dongzhu Liu, Osvaldo Simeone, Guangxu Zhu
• Abstract summary: We propose wireless FALD, a protocol that realizes FALD in wireless systems. WFALD integrates over-the-air computation and channel-driven sampling for Monte Carlo updates. Analysis and experiments show that, when the signal-to-noise ratio is sufficiently large, channel noise can be fully repurposed for Monte Carlo sampling.
• Score: 41.58760966569499
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The recent development of scalable Bayesian inference methods has renewed interest in the adoption of Bayesian learning as an alternative to conventional frequentist learning that offers improved model calibration via uncertainty quantification. Recently, federated averaging Langevin dynamics (FALD) was introduced as a variant of federated averaging that can efficiently implement distributed Bayesian learning in the presence of noiseless communications. In this paper, we propose wireless FALD (WFALD), a novel protocol that realizes FALD in wireless systems by integrating over-the-air computation and channel-driven sampling for Monte Carlo updates. Unlike prior work on wireless Bayesian learning, WFALD enables (\emph{i}) multiple local updates between communication rounds; and (\emph{ii}) stochastic gradients computed by mini-batch. A convergence analysis is presented in terms of the 2-Wasserstein distance between the samples produced by WFALD and the targeted global posterior distribution. Analysis and experiments show that, when the signal-to-noise ratio is sufficiently large, channel noise can be fully repurposed for Monte Carlo sampling, thus entailing no loss in performance.

Related papers

• A Universal Deep Neural Network for Signal Detection in Wireless Communication Systems [35.07773969966621]
  Deep learning (DL) has been emerging as a promising approach for channel estimation and signal detection in wireless communications. To cope with the dynamic nature of the wireless channel, DL methods must be re-trained on newly non-aged collected data. This paper proposes a novel universal deep neural network (Uni-DNN) that can achieve high detection performance in various wireless environments without retraining the model.
  arXiv  Detail & Related papers  (2024-04-03T11:21:10Z)
• Over-the-Air Federated Learning and Optimization [52.5188988624998]
  We focus on Federated learning (FL) via edge-the-air computation (AirComp) We describe the convergence of AirComp-based FedAvg (AirFedAvg) algorithms under both convex and non- convex settings. For different types of local updates that can be transmitted by edge devices (i.e., model, gradient, model difference), we reveal that transmitting in AirFedAvg may cause an aggregation error. In addition, we consider more practical signal processing schemes to improve the communication efficiency and extend the convergence analysis to different forms of model aggregation error caused by these signal processing schemes.
  arXiv  Detail & Related papers  (2023-10-16T05:49:28Z)
• Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization [87.21285093582446]
  Diffusion Generative Flow Samplers (DGFS) is a sampling-based framework where the learning process can be tractably broken down into short partial trajectory segments. Our method takes inspiration from the theory developed for generative flow networks (GFlowNets)
  arXiv  Detail & Related papers  (2023-10-04T09:39:05Z)
• Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion [56.38386580040991]
  Consistency Trajectory Model (CTM) is a generalization of Consistency Models (CM) CTM enables the efficient combination of adversarial training and denoising score matching loss to enhance performance. Unlike CM, CTM's access to the score function can streamline the adoption of established controllable/conditional generation methods.
  arXiv  Detail & Related papers  (2023-10-01T05:07:17Z)
• Score-based Source Separation with Applications to Digital Communication Signals [72.6570125649502]
  We propose a new method for separating superimposed sources using diffusion-based generative models. Motivated by applications in radio-frequency (RF) systems, we are interested in sources with underlying discrete nature. Our method can be viewed as a multi-source extension to the recently proposed score distillation sampling scheme.
  arXiv  Detail & Related papers  (2023-06-26T04:12:40Z)
• Hybrid Far- and Near-Field Channel Estimation for THz Ultra-Massive MIMO via Fixed Point Networks [15.498866529344275]
  Terahertz ultra-massive multiple-input multiple-output (THz UM-MIMO) is envisioned as one of the key enablers of 6G wireless systems. We develop an efficient deep learning based channel estimator with adaptive complexity and linear convergence guarantee. A major algorithmic innovation involves applying fixed point to compute the channel estimate while modeling neural networks with arbitrary depth and adapting to the hybrid-field channel conditions.
  arXiv  Detail & Related papers  (2022-05-10T14:57:56Z)
• Low Complexity Channel estimation with Neural Network Solutions [1.0499453838486013]
  We deploy a general residual convolutional neural network to achieve channel estimation in a downlink scenario. Compared with other deep learning methods for channel estimation, our results suggest improved mean squared error computation.
  arXiv  Detail & Related papers  (2022-01-24T19:55:10Z)
• On Convergence of Federated Averaging Langevin Dynamics [22.013125418713763]
  We propose a federated averaging Langevin algorithm (FA-LD) for uncertainty quantification and mean predictions with distributed clients. We develop theoretical guarantees for FA-LD for strongly log-con distributions with non-icaved data. We show convergence results based on different averaging schemes where only partial device updates are available.
  arXiv  Detail & Related papers  (2021-12-09T18:54:29Z)
• Deep Diffusion Models for Robust Channel Estimation [1.7259824817932292]
  We introduce a novel approach for multiple-input multiple-output (MIMO) channel estimation using deep diffusion models. Our method uses a deep neural network that is trained to estimate the gradient of the log-likelihood of wireless channels at any point in high-dimensional space.
  arXiv  Detail & Related papers  (2021-11-16T01:32:11Z)
• A Generative Learning Approach for Spatio-temporal Modeling in Connected Vehicular Network [55.852401381113786]
  This paper proposes LaMI (Latency Model Inpainting), a novel framework to generate a comprehensive-temporal quality framework for wireless access latency of connected vehicles. LaMI adopts the idea from image inpainting and synthesizing and can reconstruct the missing latency samples by a two-step procedure. In particular, it first discovers the spatial correlation between samples collected in various regions using a patching-based approach and then feeds the original and highly correlated samples into a Varienational Autocoder (VAE)
  arXiv  Detail & Related papers  (2020-03-16T03:43:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.