Joint Optimization of an Autoencoder for Clustering and Embedding

• URL: http://arxiv.org/abs/2012.03740v2
• Date: Sat, 1 May 2021 20:24:34 GMT
• Title: Joint Optimization of an Autoencoder for Clustering and Embedding
• Authors: Ahc\`ene Boubekki, Michael Kampffmeyer, Robert Jenssen, Ulf Brefeld
• Abstract summary: We present an alternative where the autoencoder and the clustering are learned simultaneously. That simple neural network, referred to as the clustering module, can be integrated into a deep autoencoder resulting in a deep clustering model.
• Score: 22.16059261437617
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep embedded clustering has become a dominating approach to unsupervised categorization of objects with deep neural networks. The optimization of the most popular methods alternates between the training of a deep autoencoder and a k-means clustering of the autoencoder's embedding. The diachronic setting, however, prevents the former to benefit from valuable information acquired by the latter. In this paper, we present an alternative where the autoencoder and the clustering are learned simultaneously. This is achieved by providing novel theoretical insight, where we show that the objective function of a certain class of Gaussian mixture models (GMMs) can naturally be rephrased as the loss function of a one-hidden layer autoencoder thus inheriting the built-in clustering capabilities of the GMM. That simple neural network, referred to as the clustering module, can be integrated into a deep autoencoder resulting in a deep clustering model able to jointly learn a clustering and an embedding. Experiments confirm the equivalence between the clustering module and Gaussian mixture models. Further evaluations affirm the empirical relevance of our deep architecture as it outperforms related baselines on several data sets.

Related papers

• Transferable Deep Clustering Model [14.073783373395196]
  We propose a novel transferable deep clustering model that can automatically adapt the cluster centroids according to the distribution of data samples. Our approach introduces a novel attention-based module that can adapt the centroids by measuring their relationship with samples. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness and efficiency of our proposed transfer learning framework.
  arXiv  Detail & Related papers  (2023-10-07T23:35:17Z)
• Reinforcement Graph Clustering with Unknown Cluster Number [91.4861135742095]
  We propose a new deep graph clustering method termed Reinforcement Graph Clustering. In our proposed method, cluster number determination and unsupervised representation learning are unified into a uniform framework. In order to conduct feedback actions, the clustering-oriented reward function is proposed to enhance the cohesion of the same clusters and separate the different clusters.
  arXiv  Detail & Related papers  (2023-08-13T18:12:28Z)
• DeepCluE: Enhanced Image Clustering via Multi-layer Ensembles in Deep Neural Networks [53.88811980967342]
  This paper presents a Deep Clustering via Ensembles (DeepCluE) approach. It bridges the gap between deep clustering and ensemble clustering by harnessing the power of multiple layers in deep neural networks. Experimental results on six image datasets confirm the advantages of DeepCluE over the state-of-the-art deep clustering approaches.
  arXiv  Detail & Related papers  (2022-06-01T09:51:38Z)
• Deep clustering with fusion autoencoder [0.0]
  Deep clustering (DC) models capitalize on autoencoders to learn intrinsic features which facilitate the clustering process in consequence. In this paper, a novel DC method is proposed to address this issue. Specifically, the generative adversarial network and VAE are coalesced into a new autoencoder called fusion autoencoder (FAE)
  arXiv  Detail & Related papers  (2022-01-11T07:38:03Z)
• Deep Attention-guided Graph Clustering with Dual Self-supervision [49.040136530379094]
  We propose a novel method, namely deep attention-guided graph clustering with dual self-supervision (DAGC) We develop a dual self-supervision solution consisting of a soft self-supervision strategy with a triplet Kullback-Leibler divergence loss and a hard self-supervision strategy with a pseudo supervision loss. Our method consistently outperforms state-of-the-art methods on six benchmark datasets.
  arXiv  Detail & Related papers  (2021-11-10T06:53:03Z)
• Mixture Model Auto-Encoders: Deep Clustering through Dictionary Learning [72.9458277424712]
  Mixture Model Auto-Encoders (MixMate) is a novel architecture that clusters data by performing inference on a generative model. We show that MixMate achieves competitive performance compared to state-of-the-art deep clustering algorithms.
  arXiv  Detail & Related papers  (2021-10-10T02:30:31Z)
• Attention-driven Graph Clustering Network [49.040136530379094]
  We propose a novel deep clustering method named Attention-driven Graph Clustering Network (AGCN) AGCN exploits a heterogeneous-wise fusion module to dynamically fuse the node attribute feature and the topological graph feature. AGCN can jointly perform feature learning and cluster assignment in an unsupervised fashion.
  arXiv  Detail & Related papers  (2021-08-12T02:30:38Z)
• Augmented Data as an Auxiliary Plug-in Towards Categorization of Crowdsourced Heritage Data [2.609784101826762]
  We propose a strategy to mitigate the problem of inefficient clustering performance by introducing data augmentation as an auxiliary plug-in. We train a variant of Convolutional Autoencoder (CAE) with augmented data to construct the initial feature space as a novel model for deep clustering.
  arXiv  Detail & Related papers  (2021-07-08T14:09:39Z)
• Deep adaptive fuzzy clustering for evolutionary unsupervised representation learning [2.8028128734158164]
  Cluster assignment of large and complex images is a crucial but challenging task in pattern recognition and computer vision. We present a novel evolutionary unsupervised learning representation model with iterative optimization. We jointly fuzzy clustering to the deep reconstruction model, in which fuzzy membership is utilized to represent a clear structure of deep cluster assignments.
  arXiv  Detail & Related papers  (2021-03-31T13:58:10Z)
• Scalable Hierarchical Agglomerative Clustering [65.66407726145619]
  Existing scalable hierarchical clustering methods sacrifice quality for speed. We present a scalable, agglomerative method for hierarchical clustering that does not sacrifice quality and scales to billions of data points.
  arXiv  Detail & Related papers  (2020-10-22T15:58:35Z)
• Dissimilarity Mixture Autoencoder for Deep Clustering [0.0]
  The dissimilarity mixture autoencoder (DMAE) is a neural network model for feature-based clustering. DMAE can be integrated with deep learning architectures into end-to-end models.
  arXiv  Detail & Related papers  (2020-06-15T07:08:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.