Online Deep Clustering for Unsupervised Representation Learning

• URL: http://arxiv.org/abs/2006.10645v1
• Date: Thu, 18 Jun 2020 16:15:46 GMT
• Title: Online Deep Clustering for Unsupervised Representation Learning
• Authors: Xiaohang Zhan, Jiahao Xie, Ziwei Liu, Yew Soon Ong, Chen Change Loy
• Abstract summary: Online Deep Clustering (ODC) performs clustering and network update simultaneously rather than alternatingly. We design and maintain two dynamic memory modules, i.e., samples memory to store samples labels and features, and centroids memory for centroids evolution. In this way, labels and the network evolve shoulder-to-shoulder rather than alternatingly.
• Score: 108.33534231219464
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Joint clustering and feature learning methods have shown remarkable performance in unsupervised representation learning. However, the training schedule alternating between feature clustering and network parameters update leads to unstable learning of visual representations. To overcome this challenge, we propose Online Deep Clustering (ODC) that performs clustering and network update simultaneously rather than alternatingly. Our key insight is that the cluster centroids should evolve steadily in keeping the classifier stably updated. Specifically, we design and maintain two dynamic memory modules, i.e., samples memory to store samples labels and features, and centroids memory for centroids evolution. We break down the abrupt global clustering into steady memory update and batch-wise label re-assignment. The process is integrated into network update iterations. In this way, labels and the network evolve shoulder-to-shoulder rather than alternatingly. Extensive experiments demonstrate that ODC stabilizes the training process and boosts the performance effectively. Code: https://github.com/open-mmlab/OpenSelfSup.

Related papers

• Reinforcement Graph Clustering with Unknown Cluster Number [91.4861135742095]
  We propose a new deep graph clustering method termed Reinforcement Graph Clustering. In our proposed method, cluster number determination and unsupervised representation learning are unified into a uniform framework. In order to conduct feedback actions, the clustering-oriented reward function is proposed to enhance the cohesion of the same clusters and separate the different clusters.
  arXiv  Detail & Related papers  (2023-08-13T18:12:28Z)
• Hard Regularization to Prevent Deep Online Clustering Collapse without Data Augmentation [65.268245109828]
  Online deep clustering refers to the joint use of a feature extraction network and a clustering model to assign cluster labels to each new data point or batch as it is processed. While faster and more versatile than offline methods, online clustering can easily reach the collapsed solution where the encoder maps all inputs to the same point and all are put into a single cluster. We propose a method that does not require data augmentation, and that, differently from existing methods, regularizes the hard assignments.
  arXiv  Detail & Related papers  (2023-03-29T08:23:26Z)
• Dual Cluster Contrastive learning for Person Re-Identification [78.42770787790532]
  We formulate a unified cluster contrastive framework, named Dual Cluster Contrastive learning (DCC) DCC maintains two types of memory banks: individual and centroid cluster memory banks. It can be easily applied for unsupervised or supervised person ReID.
  arXiv  Detail & Related papers  (2021-12-09T02:43:25Z)
• Unsupervised Person Re-identification with Stochastic Training Strategy [29.639040901941726]
  State-of-the-art unsupervised re-ID methods usually follow a clustering-based strategy. Forcing images to get closer to the centroid emphasizes the result of clustering. Previous methods utilize features obtained at different training iterations to represent one centroid. We propose an unsupervised re-ID approach with a learning strategy.
  arXiv  Detail & Related papers  (2021-08-16T07:23:58Z)
• Selective Pseudo-label Clustering [42.19193184852487]
  Deep neural networks (DNNs) offer a means of addressing the challenging task of clustering high-dimensional data. We propose selective pseudo-label clustering, which uses only the most confident pseudo-labels for training theDNN. New approach achieves a state-of-the-art performance on three popular image datasets.
  arXiv  Detail & Related papers  (2021-07-22T13:56:53Z)
• You Never Cluster Alone [150.94921340034688]
  We extend the mainstream contrastive learning paradigm to a cluster-level scheme, where all the data subjected to the same cluster contribute to a unified representation. We define a set of categorical variables as clustering assignment confidence, which links the instance-level learning track with the cluster-level one. By reparametrizing the assignment variables, TCC is trained end-to-end, requiring no alternating steps.
  arXiv  Detail & Related papers  (2021-06-03T14:59:59Z)
• Unsupervised Visual Representation Learning by Online Constrained K-Means [44.38989920488318]
  Cluster discrimination is an effective pretext task for unsupervised representation learning. We propose a novel clustering-based pretext task with online textbfConstrained textbfK-mtextbfeans (textbfCoKe) Our online assignment method has a theoretical guarantee to approach the global optimum.
  arXiv  Detail & Related papers  (2021-05-24T20:38:32Z)
• Unsupervised Person Re-identification via Softened Similarity Learning [122.70472387837542]
  Person re-identification (re-ID) is an important topic in computer vision. This paper studies the unsupervised setting of re-ID, which does not require any labeled information. Experiments on two image-based and video-based datasets demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-04-07T17:16:41Z)
• GATCluster: Self-Supervised Gaussian-Attention Network for Image Clustering [9.722607434532883]
  We propose a self-supervised clustering network for image Clustering (GATCluster) Rather than extracting intermediate features first and then performing the traditional clustering, GATCluster semantic cluster labels without further post-processing. We develop a two-step learning algorithm that is memory-efficient for clustering large-size images.
  arXiv  Detail & Related papers  (2020-02-27T00:57:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.