Common Failure Modes of Subcluster-based Sampling in Dirichlet Process
Gaussian Mixture Models -- and a Deep-learning Solution
- URL: http://arxiv.org/abs/2203.13661v1
- Date: Fri, 25 Mar 2022 14:12:33 GMT
- Title: Common Failure Modes of Subcluster-based Sampling in Dirichlet Process
Gaussian Mixture Models -- and a Deep-learning Solution
- Authors: Vlad Winter, Or Dinari, Oren Freifeld
- Abstract summary: Dirichlet Process Gaussian Mixture Model (DPGMM) is often used to cluster data when the number of clusters is unknown.
One main DPGMM inference paradigm relies on sampling.
Here we consider a known state-of-art sampler, analyze its failure modes, and show how to improve it.
- Score: 5.822529963339041
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The Dirichlet Process Gaussian Mixture Model (DPGMM) is often used to cluster
data when the number of clusters is unknown. One main DPGMM inference paradigm
relies on sampling. Here we consider a known state-of-art sampler (proposed by
Chang and Fisher III (2013) and improved by Dinari et al. (2019)), analyze its
failure modes, and show how to improve it, often drastically. Concretely, in
that sampler, whenever a new cluster is formed it is augmented with two
subclusters whose labels are initialized at random. Upon their evolution, the
subclusters serve to propose a split of the parent cluster. We show that the
random initialization is often problematic and hurts the otherwise-effective
sampler. Specifically, we demonstrate that this initialization tends to lead to
poor split proposals and/or too many iterations before a desired split is
accepted. This slows convergence and can damage the clustering. As a remedy, we
propose two drop-in-replacement options for the subcluster-initialization
subroutine. The first is an intuitive heuristic while the second is based on
deep learning. We show that the proposed approach yields better splits, which
in turn translate to substantial improvements in performance, results, and
stability.
Related papers
- Adaptive Intra-Class Variation Contrastive Learning for Unsupervised Person Re-Identification [10.180143197144803]
We propose an adaptive intra-class variation contrastive learning algorithm for unsupervised Re-ID, called AdaInCV.
The algorithm quantitatively evaluates the learning ability of the model for each class by considering the intra-class variations after clustering.
To be more specific, two new strategies are proposed: Adaptive Sample Mining (AdaSaM) and Adaptive Outlier Filter (AdaOF)
arXiv Detail & Related papers (2024-04-06T15:48:14Z) - Fast Semisupervised Unmixing Using Nonconvex Optimization [80.11512905623417]
We introduce a novel convex convex model for semi/library-based unmixing.
We demonstrate the efficacy of Alternating Methods of sparse unsupervised unmixing.
arXiv Detail & Related papers (2024-01-23T10:07:41Z) - A provable initialization and robust clustering method for general mixture models [6.806940901668607]
Clustering is a fundamental tool in statistical machine learning in the presence of heterogeneous data.
Most recent results focus on optimal mislabeling guarantees when data are distributed around centroids with sub-Gaussian errors.
arXiv Detail & Related papers (2024-01-10T22:56:44Z) - Compound Batch Normalization for Long-tailed Image Classification [77.42829178064807]
We propose a compound batch normalization method based on a Gaussian mixture.
It can model the feature space more comprehensively and reduce the dominance of head classes.
The proposed method outperforms existing methods on long-tailed image classification.
arXiv Detail & Related papers (2022-12-02T07:31:39Z) - Implicit Sample Extension for Unsupervised Person Re-Identification [97.46045935897608]
Clustering sometimes mixes different true identities together or splits the same identity into two or more sub clusters.
We propose an Implicit Sample Extension (OurWholeMethod) method to generate what we call support samples around the cluster boundaries.
Experiments demonstrate that the proposed method is effective and achieves state-of-the-art performance for unsupervised person Re-ID.
arXiv Detail & Related papers (2022-04-14T11:41:48Z) - Personalized Federated Learning via Convex Clustering [72.15857783681658]
We propose a family of algorithms for personalized federated learning with locally convex user costs.
The proposed framework is based on a generalization of convex clustering in which the differences between different users' models are penalized.
arXiv Detail & Related papers (2022-02-01T19:25:31Z) - Correlation Clustering Reconstruction in Semi-Adversarial Models [70.11015369368272]
Correlation Clustering is an important clustering problem with many applications.
We study the reconstruction version of this problem in which one is seeking to reconstruct a latent clustering corrupted by random noise and adversarial modifications.
arXiv Detail & Related papers (2021-08-10T14:46:17Z) - Progressive Cluster Purification for Unsupervised Feature Learning [48.87365358296371]
In unsupervised feature learning, sample specificity based methods ignore the inter-class information.
We propose a novel clustering based method, which excludes class inconsistent samples during progressive cluster formation.
Our approach, referred to as Progressive Cluster Purification (PCP), implements progressive clustering by gradually reducing the number of clusters during training.
arXiv Detail & Related papers (2020-07-06T08:11:03Z) - An Efficient Framework for Clustered Federated Learning [26.24231986590374]
We address the problem of federated learning (FL) where users are distributed into clusters.
We propose the Iterative Federated Clustering Algorithm (IFCA)
We show that our algorithm is efficient in non- partitioned problems such as neural networks.
arXiv Detail & Related papers (2020-06-07T08:48:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.