Federated Variational Inference for Bayesian Mixture Models

• URL: http://arxiv.org/abs/2502.12684v1
• Date: Tue, 18 Feb 2025 09:41:23 GMT
• Title: Federated Variational Inference for Bayesian Mixture Models
• Abstract summary: We introduce a principled 'divide and conquer' inference procedure using variational inference with local merge and delete moves.<n>We show that these merge moves require only summaries of the data in each batch, enabling federated learning across local nodes.<n>We validate the practical utility of the method by applying it to large scale electronic health record (EHR) data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a federated learning approach for Bayesian model-based clustering of large-scale binary and categorical datasets. We introduce a principled 'divide and conquer' inference procedure using variational inference with local merge and delete moves within batches of the data in parallel, followed by 'global' merge moves across batches to find global clustering structures. We show that these merge moves require only summaries of the data in each batch, enabling federated learning across local nodes without requiring the full dataset to be shared. Empirical results on simulated and benchmark datasets demonstrate that our method performs well in comparison to existing clustering algorithms. We validate the practical utility of the method by applying it to large scale electronic health record (EHR) data.

Related papers

• Imputation Meets Clustering: Exploiting Latent Subgroup Structure for Missing Data Recovery [52.29866804948133]
  We propose CAGI (Cluster-Aware Generative Imputation), a framework that reformulates clustering and imputation as a mutually reinforcing co-optimization process.<n>Experiments on 14 benchmark datasets with 15 representative baselines demonstrate the superiority of CAGI.
  arXiv  Detail & Related papers  (2026-07-08T02:50:02Z)
• Variational Consensus Monte Carlo for Bayesian Mixture [0.0]
  Motivated by the privacy, sensitivity and sharing limitations of health data, we present a comprehensive pipeline for inference of Bayesian mixture models.<n>We adopt a Consensus Monte Carlo approach, in which an MCMC algorithm is run independently within each data silo to estimate local posterior distributions.<n>A comprehensive simulation study validates the framework and allows us to compare to state-of-the-art federated learning alternatives.
  arXiv  Detail & Related papers  (2026-06-17T22:48:43Z)
• Semantic-based Distributed Learning for Diverse and Discriminative Representations [26.214742067718277]
  In large-scale distributed scenarios, increasingly complex tasks demand more intelligent collaboration across networks.<n>We propose a novel distributed learning framework that ensures both diverse and discriminative representations.<n>In both cases, the resulting optimal solutions are theoretically proven to maintain discriminative and diverse properties.
  arXiv  Detail & Related papers  (2026-04-20T13:22:58Z)
• Weight-Informed Self-Explaining Clustering for Mixed-Type Tabular Data [63.62853416081748]
  WISE is a framework that unifies representation, feature weighting, clustering, and interpretation.<n>It produces faithful, human-interpretable explanations grounded in the same primitives that drive clustering.
  arXiv  Detail & Related papers  (2026-04-07T13:18:31Z)
• Bridging the Semantic Gap for Categorical Data Clustering via Large Language Models [64.58262227709842]
  ARISE (Attention-weighted Representation with Integrated Semantic Embeddings) is presented.<n>It builds semantic-aware representations that complement the metric space of categorical data for accurate clustering.<n>Experiments on eight benchmark datasets demonstrate consistent improvements over seven representative counterparts.
  arXiv  Detail & Related papers  (2026-01-03T11:37:46Z)
• A new type of federated clustering: A non-model-sharing approach [6.5347053452025206]
  This study proposes data collaboration clustering (DC-Clustering)<n>DC-Clustering supports clustering over complex data partitioning scenarios where horizontal and vertical splits coexist.<n>Results show that our method achieves clustering performance comparable to centralized clustering where all data are pooled.
  arXiv  Detail & Related papers  (2025-06-11T23:57:26Z)
• Federated Clustering: An Unsupervised Cluster-Wise Training for Decentralized Data Distributions [1.6385815610837167]
  Federated Cluster-Wise Refinement (FedCRef) involves clients that collaboratively train models on clusters with similar data distributions. In these groups, clients collaboratively train a shared model representing each data distribution, while continuously refining their local clusters to enhance data association accuracy. This iterative process allows our system to identify all potential data distributions across the network and develop robust representation models for each.
  arXiv  Detail & Related papers  (2024-08-20T09:05:44Z)
• Interpetable Target-Feature Aggregation for Multi-Task Learning based on Bias-Variance Analysis [53.38518232934096]
  Multi-task learning (MTL) is a powerful machine learning paradigm designed to leverage shared knowledge across tasks to improve generalization and performance. We propose an MTL approach at the intersection between task clustering and feature transformation based on a two-phase iterative aggregation of targets and features. In both phases, a key aspect is to preserve the interpretability of the reduced targets and features through the aggregation with the mean, which is motivated by applications to Earth science.
  arXiv  Detail & Related papers  (2024-06-12T08:30:16Z)
• Multiply Robust Estimation for Local Distribution Shifts with Multiple Domains [9.429772474335122]
  We focus on scenarios where data distributions vary across multiple segments of the entire population. We propose a two-stage multiply robust estimation method to improve model performance on each individual segment. Our method is designed to be implemented with commonly used off-the-shelf machine learning models.
  arXiv  Detail & Related papers  (2024-02-21T22:01:10Z)
• Dynamically Weighted Federated k-Means [0.0]
  Federated clustering enables multiple data sources to collaboratively cluster their data, maintaining decentralization and preserving privacy. We introduce a novel federated clustering algorithm named Dynamically Weighted Federated k-means (DWF k-means) based on Lloyd's method for k-means clustering. We conduct experiments on multiple datasets and data distribution settings to evaluate the performance of our algorithm in terms of clustering score, accuracy, and v-measure.
  arXiv  Detail & Related papers  (2023-10-23T12:28:21Z)
• Integrating Local Real Data with Global Gradient Prototypes for Classifier Re-Balancing in Federated Long-Tailed Learning [60.41501515192088]
  Federated Learning (FL) has become a popular distributed learning paradigm that involves multiple clients training a global model collaboratively. The data samples usually follow a long-tailed distribution in the real world, and FL on the decentralized and long-tailed data yields a poorly-behaved global model. In this work, we integrate the local real data with the global gradient prototypes to form the local balanced datasets.
  arXiv  Detail & Related papers  (2023-01-25T03:18:10Z)
• Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning [112.69497636932955]
  Federated learning aims to train models across different clients without the sharing of data for privacy considerations. We study how data heterogeneity affects the representations of the globally aggregated models. We propose sc FedDecorr, a novel method that can effectively mitigate dimensional collapse in federated learning.
  arXiv  Detail & Related papers  (2022-10-01T09:04:17Z)
• DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup [58.894901088797376]
  federated learning has emerged as a promising approach for training a global model using data from multiple organizations without leaking their raw data. We propose a general framework to solve the above two challenges simultaneously. We provide comprehensive theoretical analysis including robustness analysis, convergence analysis, and generalization ability.
  arXiv  Detail & Related papers  (2022-04-16T08:08:29Z)
• Deep Conditional Gaussian Mixture Model for Constrained Clustering [7.070883800886882]
  Constrained clustering can leverage prior information on a growing amount of only partially labeled data. We propose a novel framework for constrained clustering that is intuitive, interpretable, and can be trained efficiently in the framework of gradient variational inference.
  arXiv  Detail & Related papers  (2021-06-11T13:38:09Z)
• A Multiscale Environment for Learning by Diffusion [9.619814126465206]
  We introduce the Multiscale Environment for Learning by Diffusion (MELD) data model. We show that the MELD data model precisely captures latent multiscale structure in data and facilitates its analysis. To efficiently learn the multiscale structure observed in many real datasets, we introduce the Multiscale Learning by Unsupervised Diffusion (M-LUND) clustering algorithm.
  arXiv  Detail & Related papers  (2021-01-31T17:46:19Z)
• Model Fusion with Kullback--Leibler Divergence [58.20269014662046]
  We propose a method to fuse posterior distributions learned from heterogeneous datasets. Our algorithm relies on a mean field assumption for both the fused model and the individual dataset posteriors.
  arXiv  Detail & Related papers  (2020-07-13T03:27:45Z)
• LSD-C: Linearly Separable Deep Clusters [145.89790963544314]
  We present LSD-C, a novel method to identify clusters in an unlabeled dataset. Our method draws inspiration from recent semi-supervised learning practice and proposes to combine our clustering algorithm with self-supervised pretraining and strong data augmentation. We show that our approach significantly outperforms competitors on popular public image benchmarks including CIFAR 10/100, STL 10 and MNIST, as well as the document classification dataset Reuters 10K.
  arXiv  Detail & Related papers  (2020-06-17T17:58:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.