Transformers can do Bayesian Clustering

• URL: http://arxiv.org/abs/2510.24318v1
• Date: Tue, 28 Oct 2025 11:36:31 GMT
• Title: Transformers can do Bayesian Clustering
• Abstract summary: We present Cluster-PFN, a Transformer-based model that extends Prior-Data Fitted Networks (PFNs) to unsupervised Bayesian clustering.<n> trained entirely on synthetic datasets generated from a finite Gaussian Mixture Model (GMM) prior, Cluster-PFN learns to estimate the posterior distribution over both the number of clusters and the cluster assignments.<n> Cluster-PFN can be trained on complex priors that include missing data, outperforming imputation-based baselines on real-world genomic datasets, at high missingness.
• Score: 0.2578242050187029
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bayesian clustering accounts for uncertainty but is computationally demanding at scale. Furthermore, real-world datasets often contain missing values, and simple imputation ignores the associated uncertainty, resulting in suboptimal results. We present Cluster-PFN, a Transformer-based model that extends Prior-Data Fitted Networks (PFNs) to unsupervised Bayesian clustering. Trained entirely on synthetic datasets generated from a finite Gaussian Mixture Model (GMM) prior, Cluster-PFN learns to estimate the posterior distribution over both the number of clusters and the cluster assignments. Our method estimates the number of clusters more accurately than handcrafted model selection procedures such as AIC, BIC and Variational Inference (VI), and achieves clustering quality competitive with VI while being orders of magnitude faster. Cluster-PFN can be trained on complex priors that include missing data, outperforming imputation-based baselines on real-world genomic datasets, at high missingness. These results show that the Cluster-PFN can provide scalable and flexible Bayesian clustering.

Related papers

• Imputation Meets Clustering: Exploiting Latent Subgroup Structure for Missing Data Recovery [52.29866804948133]
  We propose CAGI (Cluster-Aware Generative Imputation), a framework that reformulates clustering and imputation as a mutually reinforcing co-optimization process.<n>Experiments on 14 benchmark datasets with 15 representative baselines demonstrate the superiority of CAGI.
  arXiv  Detail & Related papers  (2026-07-08T02:50:02Z)
• TabClustPFN: A Prior-Fitted Network for Tabular Data Clustering [11.86976498650059]
  We introduce TabClustPFN, a prior-fitted network for data clustering.<n>It performs amortized Bayesian inference over both cluster assignments and cluster cardinality.<n>It outperforms classical, deep, and amortized clustering baselines.
  arXiv  Detail & Related papers  (2026-01-29T12:56:41Z)
• Robust Categorical Data Clustering Guided by Multi-Granular Competitive Learning [47.32771052588132]
  The nested granular cluster effect is prevalent in the implicit discrete distance space of categorical data.<n>We propose a Multi-Granular Competitiveization Learning algorithm to allow potential clusters to interactively tune themselves.<n>It is shown that the proposed MGCPL-guided Categorical Data Clustering approach is competent in exploring the nested distribution of multi-granular clusters.
  arXiv  Detail & Related papers  (2026-01-23T06:33:08Z)
• Clustering by Attention: Leveraging Prior Fitted Transformers for Data Partitioning [3.4530027457862005]
  We introduce a novel clustering approach based on meta-learning.<n>We employ a pre-trained Prior-Data Fitted Transformer Network (PFN) to perform clustering.<n>We show that our approach is superior to the state-of-the-art clustering techniques.
  arXiv  Detail & Related papers  (2025-07-27T17:53:19Z)
• CLoVE: Personalized Federated Learning through Clustering of Loss Vector Embeddings [1.966764032092535]
  We propose CLoVE, a novel algorithm for Clustered Federated Learning (CFL)<n>CLoVE utilizes client embeddings derived from model losses on client data, and leverages the insight that clients in the same cluster share similar loss values.<n>CLoVE is able to iteratively identify and separate clients from different clusters and optimize cluster-specific models.
  arXiv  Detail & Related papers  (2025-06-27T17:52:16Z)
• Towards Learnable Anchor for Deep Multi-View Clustering [49.767879678193005]
  In this paper, we propose the Deep Multi-view Anchor Clustering (DMAC) model that performs clustering in linear time.<n>With the optimal anchors, the full sample graph is calculated to derive a discriminative embedding for clustering.<n>Experiments on several datasets demonstrate superior performance and efficiency of DMAC compared to state-of-the-art competitors.
  arXiv  Detail & Related papers  (2025-03-16T09:38:11Z)
• Consistent Amortized Clustering via Generative Flow Networks [6.9158153233702935]
  We introduce GFNCP, a novel framework for amortized clustering.<n> GFNCP is formulated as a Generative Flow Network with a shared energy-based parametrization of policy and reward.<n>We show that the flow matching conditions are equivalent to consistency of the clustering posterior under marginalization.
  arXiv  Detail & Related papers  (2025-02-26T17:30:52Z)
• Interaction-Aware Gaussian Weighting for Clustered Federated Learning [58.92159838586751]
  Federated Learning (FL) emerged as a decentralized paradigm to train models while preserving privacy.<n>We propose a novel clustered FL method, FedGWC (Federated Gaussian Weighting Clustering), which groups clients based on their data distribution.<n>Our experiments on benchmark datasets show that FedGWC outperforms existing FL algorithms in cluster quality and classification accuracy.
  arXiv  Detail & Related papers  (2025-02-05T16:33:36Z)
• UniForCE: The Unimodality Forest Method for Clustering and Estimation of the Number of Clusters [2.4953699842881605]
  We focus on the concept of unimodality and propose a flexible cluster definition called locally unimodal cluster. A locally unimodal cluster extends for as long as unimodality is locally preserved across pairs of subclusters of the data. We propose the UniForCE method for locally unimodal clustering.
  arXiv  Detail & Related papers  (2023-12-18T16:19:02Z)
• Revisiting Instance-Optimal Cluster Recovery in the Labeled Stochastic Block Model [85.51611950757643]
  We propose IAC (Instance-Adaptive Clustering), the first algorithm whose performance matches the instance-specific lower bounds both in expectation and with high probability.<n>IAC maintains an overall computational complexity of $ mathcalO(n, textpolylog(n) $, making it scalable and practical for large-scale problems.
  arXiv  Detail & Related papers  (2023-06-18T08:46:06Z)
• DeepCluE: Enhanced Image Clustering via Multi-layer Ensembles in Deep Neural Networks [53.88811980967342]
  This paper presents a Deep Clustering via Ensembles (DeepCluE) approach. It bridges the gap between deep clustering and ensemble clustering by harnessing the power of multiple layers in deep neural networks. Experimental results on six image datasets confirm the advantages of DeepCluE over the state-of-the-art deep clustering approaches.
  arXiv  Detail & Related papers  (2022-06-01T09:51:38Z)
• Neural Mixture Models with Expectation-Maximization for End-to-end Deep Clustering [0.8543753708890495]
  In this paper, we realize mixture model-based clustering with a neural network. We train the network end-to-end via batch-wise EM iterations where the forward pass acts as the E-step and the backward pass acts as the M-step. Our trained networks outperform single-stage deep clustering methods that still depend on k-means.
  arXiv  Detail & Related papers  (2021-07-06T08:00:58Z)
• You Never Cluster Alone [150.94921340034688]
  We extend the mainstream contrastive learning paradigm to a cluster-level scheme, where all the data subjected to the same cluster contribute to a unified representation. We define a set of categorical variables as clustering assignment confidence, which links the instance-level learning track with the cluster-level one. By reparametrizing the assignment variables, TCC is trained end-to-end, requiring no alternating steps.
  arXiv  Detail & Related papers  (2021-06-03T14:59:59Z)
• Scalable Hierarchical Agglomerative Clustering [65.66407726145619]
  Existing scalable hierarchical clustering methods sacrifice quality for speed. We present a scalable, agglomerative method for hierarchical clustering that does not sacrifice quality and scales to billions of data points.
  arXiv  Detail & Related papers  (2020-10-22T15:58:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.