Variational Consensus Monte Carlo for Bayesian Mixture

• URL: http://arxiv.org/abs/2606.19643v1
• Date: Wed, 17 Jun 2026 22:48:43 GMT
• Title: Variational Consensus Monte Carlo for Bayesian Mixture
• Abstract summary: Motivated by the privacy, sensitivity and sharing limitations of health data, we present a comprehensive pipeline for inference of Bayesian mixture models.<n>We adopt a Consensus Monte Carlo approach, in which an MCMC algorithm is run independently within each data silo to estimate local posterior distributions.<n>A comprehensive simulation study validates the framework and allows us to compare to state-of-the-art federated learning alternatives.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Motivated by the privacy, sensitivity and sharing limitations of health data, we present a comprehensive pipeline for inference of Bayesian mixture models within a federated learning setting, i.e. when data cannot be fully shared or pooled across compute nodes. We adopt a Consensus Monte Carlo (CMC) approach, in which an MCMC algorithm is run independently within each data silo to estimate local posterior distributions, which are then aggregated to approximate the posterior over the full data. The variational CMC approach of Rabinovich, Angelino and Jordan (2015) [1] frames the aggregation step as a variational inference problem, but their application to mixtures assumes the number of clusters and key mixture parameters to be known. Our main methodological contributions are: (i) an extension of variational CMC to over-fitted Bayesian mixture models that infer the number of clusters and all model parameters, without requiring conjugacy; (ii) novel cluster-matching algorithms suitable for cross-silo settings in which not every cluster appears in each local dataset; (iii) a number of inference strategies for the aggregation step, matched to different federated learning constraints; and (iv) guidelines for choosing among these in practice. A comprehensive simulation study validates the framework and allows us to compare to state-of-the-art federated learning alternatives. Notably, we show that when the composition of local datasets reflects the underlying clustering structure in the data, our approach can recover small clusters with greater accuracy than standard MCMC applied to the pooled data. We illustrate the framework on large-scale electronic health record data, identifying multi-morbidity patterns in a British geriatric population.

Related papers

• CuBAS: Information Geometric Curvature-Based Adaptive Sampling for Supervised Classification [52.452902154360565]
  CuBAS is an information-geometric framework for adaptive data selection in supervised classification.<n>A labeled dataset can be viewed as a statistical manifold, on which local curvature, estimated via the ratio of second to first-order observed Fisher information, faithfully encodes the geometric complexity of the data distribution.
  arXiv  Detail & Related papers  (2026-07-03T09:36:02Z)
• Turtle shell clustering: A mixture approach to discriminative clustering with applications to flow cytometry and other data [0.254890465057467]
  Generative approaches to clustering provide information on geometric properties of clusters.<n>Discriminative approaches provide boundaries between clusters.<n>We present a fully unsupervised, probabilistic, and discriminative clustering method.
  arXiv  Detail & Related papers  (2026-04-25T00:33:30Z)
• Clustering Approaches for Mixed-Type Data: A Comparative Study [0.0]
  Clustering mixed-type data is a challenge, as few existing approaches are suited for this task.<n>This study presents the state-of-the-art of these approaches and compares them using various simulation models.<n>In our experiments KAMILA, LCM, and k-prototypes exhibited the best performance, with respect to the adjusted rand index (ARI)
  arXiv  Detail & Related papers  (2025-11-24T22:18:23Z)
• Learning Discrete Bayesian Networks with Hierarchical Dirichlet Shrinkage [52.914168158222765]
  We detail a comprehensive Bayesian framework for learning DBNs.<n>We give a novel Markov chain Monte Carlo (MCMC) algorithm utilizing parallel Langevin proposals to generate exact posterior samples.<n>We apply our methodology to uncover prognostic network structure from primary breast cancer samples.
  arXiv  Detail & Related papers  (2025-09-16T17:24:35Z)
• Federated Variational Inference for Bayesian Mixture Models [0.0]
  We introduce a principled 'divide and conquer' inference procedure using variational inference with local merge and delete moves.<n>We show that these merge moves require only summaries of the data in each batch, enabling federated learning across local nodes.<n>We validate the practical utility of the method by applying it to large scale electronic health record (EHR) data.
  arXiv  Detail & Related papers  (2025-02-18T09:41:23Z)
• Interaction-Aware Gaussian Weighting for Clustered Federated Learning [58.92159838586751]
  Federated Learning (FL) emerged as a decentralized paradigm to train models while preserving privacy.<n>We propose a novel clustered FL method, FedGWC (Federated Gaussian Weighting Clustering), which groups clients based on their data distribution.<n>Our experiments on benchmark datasets show that FedGWC outperforms existing FL algorithms in cluster quality and classification accuracy.
  arXiv  Detail & Related papers  (2025-02-05T16:33:36Z)
• A Deterministic Information Bottleneck Method for Clustering Mixed-Type Data [0.0]
  We present an information-theoretic method for clustering mixed-type data, that is, data consisting of both continuous and categorical variables.<n>The proposed approach extends the Information Bottleneck principle to heterogeneous data through generalised product kernels.<n>We demonstrate that the proposed method, named DIBmix, achieves superior performance compared to four established methods.
  arXiv  Detail & Related papers  (2024-07-03T09:06:19Z)
• Mixture of multilayer stochastic block models for multiview clustering [0.0]
  We propose an original method for aggregating multiple clustering coming from different sources of information. The identifiability of the model parameters is established and a variational Bayesian EM algorithm is proposed for the estimation of these parameters. The method is utilized to analyze global food trading networks, leading to structures of interest.
  arXiv  Detail & Related papers  (2024-01-09T17:15:47Z)
• A One-shot Framework for Distributed Clustered Learning in Heterogeneous Environments [54.172993875654015]
  The paper proposes a family of communication efficient methods for distributed learning in heterogeneous environments. One-shot approach, based on local computations at the users and a clustering based aggregation step at the server is shown to provide strong learning guarantees. For strongly convex problems it is shown that, as long as the number of data points per user is above a threshold, the proposed approach achieves order-optimal mean-squared error rates in terms of the sample size.
  arXiv  Detail & Related papers  (2022-09-22T09:04:10Z)
• Personalized Federated Learning via Convex Clustering [72.15857783681658]
  We propose a family of algorithms for personalized federated learning with locally convex user costs. The proposed framework is based on a generalization of convex clustering in which the differences between different users' models are penalized.
  arXiv  Detail & Related papers  (2022-02-01T19:25:31Z)
• Kernel learning approaches for summarising and combining posterior similarity matrices [68.8204255655161]
  We build upon the notion of the posterior similarity matrix (PSM) in order to suggest new approaches for summarising the output of MCMC algorithms for Bayesian clustering models. A key contribution of our work is the observation that PSMs are positive semi-definite, and hence can be used to define probabilistically-motivated kernel matrices.
  arXiv  Detail & Related papers  (2020-09-27T14:16:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.