Attention-Based Clustering: Learning a Kernel from Context

• URL: http://arxiv.org/abs/2010.01040v1
• Date: Fri, 2 Oct 2020 15:06:06 GMT
• Title: Attention-Based Clustering: Learning a Kernel from Context
• Authors: Samuel Coward, Erik Visse-Martindale, Chithrupa Ramesh
• Abstract summary: We propose Attention-Based Clustering (ABC), a neural architecture based on the attention mechanism. ABC is designed to learn latent representations that adapt to context within an input set. We present competitive results for clustering Omniglot characters and include analytical evidence of the effectiveness of an attention-based approach for clustering.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In machine learning, no data point stands alone. We believe that context is an underappreciated concept in many machine learning methods. We propose Attention-Based Clustering (ABC), a neural architecture based on the attention mechanism, which is designed to learn latent representations that adapt to context within an input set, and which is inherently agnostic to input sizes and number of clusters. By learning a similarity kernel, our method directly combines with any out-of-the-box kernel-based clustering approach. We present competitive results for clustering Omniglot characters and include analytical evidence of the effectiveness of an attention-based approach for clustering.

Related papers

• Self-Supervised Graph Embedding Clustering [70.36328717683297]
  K-means one-step dimensionality reduction clustering method has made some progress in addressing the curse of dimensionality in clustering tasks. We propose a unified framework that integrates manifold learning with K-means, resulting in the self-supervised graph embedding framework.
  arXiv  Detail & Related papers  (2024-09-24T08:59:51Z)
• Towards Explainable Clustering: A Constrained Declarative based Approach [0.294944680995069]
  We aim at finding a clustering that has high quality in terms of classic clustering criteria and that is explainable. A good global explanation of a clustering should give the characteristics of each cluster taking into account their abilities to describe its objects. We propose a novel interpretable constrained method called ECS for declarative computation with Explainabilty-driven Selection.
  arXiv  Detail & Related papers  (2024-03-26T21:00:06Z)
• Reinforcement Graph Clustering with Unknown Cluster Number [91.4861135742095]
  We propose a new deep graph clustering method termed Reinforcement Graph Clustering. In our proposed method, cluster number determination and unsupervised representation learning are unified into a uniform framework. In order to conduct feedback actions, the clustering-oriented reward function is proposed to enhance the cohesion of the same clusters and separate the different clusters.
  arXiv  Detail & Related papers  (2023-08-13T18:12:28Z)
• Clustering -- Basic concepts and methods [0.0]
  How can data be represented and prepared for a clustering task? How can clustering results be validated? Connectivity-based versus prototype approaches are discussed.
  arXiv  Detail & Related papers  (2022-12-01T11:40:32Z)
• Deep Clustering: A Comprehensive Survey [53.387957674512585]
  Clustering analysis plays an indispensable role in machine learning and data mining. Deep clustering, which can learn clustering-friendly representations using deep neural networks, has been broadly applied in a wide range of clustering tasks. Existing surveys for deep clustering mainly focus on the single-view fields and the network architectures, ignoring the complex application scenarios of clustering.
  arXiv  Detail & Related papers  (2022-10-09T02:31:32Z)
• KnAC: an approach for enhancing cluster analysis with background knowledge and explanations [0.20999222360659603]
  We present Knowledge Augmented Clustering (KnAC), which main goal is to confront expert-based labelling with automated clustering. KnAC can serve as an augmentation of an arbitrary clustering algorithm, making the approach robust and model-agnostic.
  arXiv  Detail & Related papers  (2021-12-16T10:13:47Z)
• Learning the Precise Feature for Cluster Assignment [39.320210567860485]
  We propose a framework which integrates representation learning and clustering into a single pipeline for the first time. The proposed framework exploits the powerful ability of recently developed generative models for learning intrinsic features. Experimental results show that the performance of the proposed method is superior, or at least comparable to, the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-11T04:08:54Z)
• Clustering-friendly Representation Learning via Instance Discrimination and Feature Decorrelation [0.0]
  We propose a clustering-friendly representation learning method using instance discrimination and feature decorrelation. In evaluations of image clustering using CIFAR-10 and ImageNet-10, our method achieves accuracy of 81.5% and 95.4%, respectively.
  arXiv  Detail & Related papers  (2021-05-31T22:59:31Z)
• Graph Contrastive Clustering [131.67881457114316]
  We propose a novel graph contrastive learning framework, which is then applied to the clustering task and we come up with the Graph Constrastive Clustering(GCC) method. Specifically, on the one hand, the graph Laplacian based contrastive loss is proposed to learn more discriminative and clustering-friendly features. On the other hand, a novel graph-based contrastive learning strategy is proposed to learn more compact clustering assignments.
  arXiv  Detail & Related papers  (2021-04-03T15:32:49Z)
• Scalable Hierarchical Agglomerative Clustering [65.66407726145619]
  Existing scalable hierarchical clustering methods sacrifice quality for speed. We present a scalable, agglomerative method for hierarchical clustering that does not sacrifice quality and scales to billions of data points.
  arXiv  Detail & Related papers  (2020-10-22T15:58:35Z)
• Structured Graph Learning for Clustering and Semi-supervised Classification [74.35376212789132]
  We propose a graph learning framework to preserve both the local and global structure of data. Our method uses the self-expressiveness of samples to capture the global structure and adaptive neighbor approach to respect the local structure. Our model is equivalent to a combination of kernel k-means and k-means methods under certain condition.
  arXiv  Detail & Related papers  (2020-08-31T08:41:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.