Deep Clustering with Diffused Sampling and Hardness-aware Self-distillation

• URL: http://arxiv.org/abs/2401.14038v1
• Date: Thu, 25 Jan 2024 09:33:49 GMT
• Title: Deep Clustering with Diffused Sampling and Hardness-aware Self-distillation
• Authors: Hai-Xin Zhang and Dong Huang
• Abstract summary: This paper proposes a novel end-to-end deep clustering method with diffused sampling and hardness-aware self-distillation (HaDis) Results on five challenging image datasets demonstrate the superior clustering performance of our HaDis method over the state-of-the-art.
• Score: 4.550555443103878
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep clustering has gained significant attention due to its capability in learning clustering-friendly representations without labeled data. However, previous deep clustering methods tend to treat all samples equally, which neglect the variance in the latent distribution and the varying difficulty in classifying or clustering different samples. To address this, this paper proposes a novel end-to-end deep clustering method with diffused sampling and hardness-aware self-distillation (HaDis). Specifically, we first align one view of instances with another view via diffused sampling alignment (DSA), which helps improve the intra-cluster compactness. To alleviate the sampling bias, we present the hardness-aware self-distillation (HSD) mechanism to mine the hardest positive and negative samples and adaptively adjust their weights in a self-distillation fashion, which is able to deal with the potential imbalance in sample contributions during optimization. Further, the prototypical contrastive learning is incorporated to simultaneously enhance the inter-cluster separability and intra-cluster compactness. Experimental results on five challenging image datasets demonstrate the superior clustering performance of our HaDis method over the state-of-the-art. Source code is available at https://github.com/Regan-Zhang/HaDis.

Related papers

• Self-Supervised Graph Embedding Clustering [70.36328717683297]
  K-means one-step dimensionality reduction clustering method has made some progress in addressing the curse of dimensionality in clustering tasks. We propose a unified framework that integrates manifold learning with K-means, resulting in the self-supervised graph embedding framework.
  arXiv  Detail & Related papers  (2024-09-24T08:59:51Z)
• GCC: Generative Calibration Clustering [55.44944397168619]
  We propose a novel Generative Clustering (GCC) method to incorporate feature learning and augmentation into clustering procedure. First, we develop a discrimirative feature alignment mechanism to discover intrinsic relationship across real and generated samples. Second, we design a self-supervised metric learning to generate more reliable cluster assignment.
  arXiv  Detail & Related papers  (2024-04-14T01:51:11Z)
• Deep Embedding Clustering Driven by Sample Stability [16.53706617383543]
  We propose a deep embedding clustering algorithm driven by sample stability (DECS) Specifically, we start by constructing the initial feature space with an autoencoder and then learn the cluster-oriented embedding feature constrained by sample stability. The experimental results on five datasets illustrate that the proposed method achieves superior performance compared to state-of-the-art clustering approaches.
  arXiv  Detail & Related papers  (2024-01-29T09:19:49Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Fairness in Visual Clustering: A Novel Transformer Clustering Approach [32.806921406869996]
  We first evaluate demographic bias in deep clustering models from the perspective of cluster purity. A novel loss function is introduced to encourage a purity consistency for all clusters to maintain the fairness aspect. We present a novel attention mechanism, Cross-attention, to measure correlations between multiple clusters.
  arXiv  Detail & Related papers  (2023-04-14T21:59:32Z)
• Self-Evolutionary Clustering [1.662966122370634]
  Most existing deep clustering methods are based on simple distance comparison and highly dependent on the target distribution generated by a handcrafted nonlinear mapping. A novel modular Self-Evolutionary Clustering (Self-EvoC) framework is constructed, which boosts the clustering performance by classification in a self-supervised manner. The framework can efficiently discriminate sample outliers and generate better target distribution with the assistance of self-supervised.
  arXiv  Detail & Related papers  (2022-02-21T19:38:18Z)
• Hybrid Dynamic Contrast and Probability Distillation for Unsupervised Person Re-Id [109.1730454118532]
  Unsupervised person re-identification (Re-Id) has attracted increasing attention due to its practical application in the read-world video surveillance system. We present the hybrid dynamic cluster contrast and probability distillation algorithm. It formulates the unsupervised Re-Id problem into an unified local-to-global dynamic contrastive learning and self-supervised probability distillation framework.
  arXiv  Detail & Related papers  (2021-09-29T02:56:45Z)
• Doubly Contrastive Deep Clustering [135.7001508427597]
  We present a novel Doubly Contrastive Deep Clustering (DCDC) framework, which constructs contrastive loss over both sample and class views. Specifically, for the sample view, we set the class distribution of the original sample and its augmented version as positive sample pairs. For the class view, we build the positive and negative pairs from the sample distribution of the class. In this way, two contrastive losses successfully constrain the clustering results of mini-batch samples in both sample and class level.
  arXiv  Detail & Related papers  (2021-03-09T15:15:32Z)
• Progressive Cluster Purification for Unsupervised Feature Learning [48.87365358296371]
  In unsupervised feature learning, sample specificity based methods ignore the inter-class information. We propose a novel clustering based method, which excludes class inconsistent samples during progressive cluster formation. Our approach, referred to as Progressive Cluster Purification (PCP), implements progressive clustering by gradually reducing the number of clusters during training.
  arXiv  Detail & Related papers  (2020-07-06T08:11:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.