Federated Variational Inference Methods for Structured Latent Variable Models

• URL: http://arxiv.org/abs/2302.03314v2
• Date: Fri, 7 Jul 2023 04:39:07 GMT
• Title: Federated Variational Inference Methods for Structured Latent Variable Models
• Authors: Conor Hassan, Robert Salomone, Kerrie Mengersen
• Abstract summary: Federated learning methods enable model training across distributed data sources without data leaving their original locations. We present a general and elegant solution based on structured variational inference, widely used in Bayesian machine learning. We also provide a communication-efficient variant analogous to the canonical FedAvg algorithm.
• Score: 1.0312968200748118
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning methods enable model training across distributed data sources without data leaving their original locations and have gained increasing interest in various fields. However, existing approaches are limited, excluding many structured probabilistic models. We present a general and elegant solution based on structured variational inference, widely used in Bayesian machine learning, adapted for the federated setting. Additionally, we provide a communication-efficient variant analogous to the canonical FedAvg algorithm. The proposed algorithms' effectiveness is demonstrated, and their performance is compared with hierarchical Bayesian neural networks and topic models.

Related papers

• Flexible inference in heterogeneous and attributed multilayer networks [21.349513661012498]
  We develop a probabilistic generative model to perform inference in multilayer networks with arbitrary types of information. We demonstrate its ability to unveil a variety of patterns in a social support network among villagers in rural India.
  arXiv  Detail & Related papers  (2024-05-31T15:21:59Z)
• BEND: Bagging Deep Learning Training Based on Efficient Neural Network Diffusion [56.9358325168226]
  We propose a Bagging deep learning training algorithm based on Efficient Neural network Diffusion (BEND) Our approach is simple but effective, first using multiple trained model weights and biases as inputs to train autoencoder and latent diffusion model. Our proposed BEND algorithm can consistently outperform the mean and median accuracies of both the original trained model and the diffused model.
  arXiv  Detail & Related papers  (2024-03-23T08:40:38Z)
• Nonparametric Automatic Differentiation Variational Inference with Spline Approximation [7.5620760132717795]
  We develop a nonparametric approximation approach that enables flexible posterior approximation for distributions with complicated structures. Compared with widely-used nonparametrical inference methods, the proposed method is easy to implement and adaptive to various data structures. Experiments demonstrate the efficiency of the proposed method in approximating complex posterior distributions and improving the performance of generative models with incomplete data.
  arXiv  Detail & Related papers  (2024-03-10T20:22:06Z)
• Implicit Variational Inference for High-Dimensional Posteriors [7.924706533725115]
  In variational inference, the benefits of Bayesian models rely on accurately capturing the true posterior distribution. We propose using neural samplers that specify implicit distributions, which are well-suited for approximating complex multimodal and correlated posteriors. Our approach introduces novel bounds for approximate inference using implicit distributions by locally linearising the neural sampler.
  arXiv  Detail & Related papers  (2023-10-10T14:06:56Z)
• Faster Adaptive Federated Learning [84.38913517122619]
  Federated learning has attracted increasing attention with the emergence of distributed data. In this paper, we propose an efficient adaptive algorithm (i.e., FAFED) based on momentum-based variance reduced technique in cross-silo FL.
  arXiv  Detail & Related papers  (2022-12-02T05:07:50Z)
• Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization [101.32332941117271]
  Decision making algorithms are used in a multitude of different applications. Deep learning approaches that use highly parametric architectures tuned from data without relying on mathematical models are becoming increasingly popular. Model-based optimization and data-centric deep learning are often considered to be distinct disciplines.
  arXiv  Detail & Related papers  (2022-05-05T13:40:08Z)
• Deep Variational Models for Collaborative Filtering-based Recommender Systems [63.995130144110156]
  Deep learning provides accurate collaborative filtering models to improve recommender system results. Our proposed models apply the variational concept to injectity in the latent space of the deep architecture. Results show the superiority of the proposed approach in scenarios where the variational enrichment exceeds the injected noise effect.
  arXiv  Detail & Related papers  (2021-07-27T08:59:39Z)
• Fully differentiable model discovery [0.0]
  We propose an approach by combining neural network based surrogates with Sparse Bayesian Learning. Our work expands PINNs to various types of neural network architectures, and connects neural network-based surrogates to the rich field of Bayesian parameter inference.
  arXiv  Detail & Related papers  (2021-06-09T08:11:23Z)
• Clustered Federated Learning via Generalized Total Variation Minimization [83.26141667853057]
  We study optimization methods to train local (or personalized) models for local datasets with a decentralized network structure. Our main conceptual contribution is to formulate federated learning as total variation minimization (GTV) Our main algorithmic contribution is a fully decentralized federated learning algorithm.
  arXiv  Detail & Related papers  (2021-05-26T18:07:19Z)
• Deep Conditional Transformation Models [0.0]
  Learning the cumulative distribution function (CDF) of an outcome variable conditional on a set of features remains challenging. Conditional transformation models provide a semi-parametric approach that allows to model a large class of conditional CDFs. We propose a novel network architecture, provide details on different model definitions and derive suitable constraints.
  arXiv  Detail & Related papers  (2020-10-15T16:25:45Z)
• Model Fusion with Kullback--Leibler Divergence [58.20269014662046]
  We propose a method to fuse posterior distributions learned from heterogeneous datasets. Our algorithm relies on a mean field assumption for both the fused model and the individual dataset posteriors.
  arXiv  Detail & Related papers  (2020-07-13T03:27:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.