Image Clustering Conditioned on Text Criteria

• URL: http://arxiv.org/abs/2310.18297v4
• Date: Thu, 22 Feb 2024 04:04:19 GMT
• Title: Image Clustering Conditioned on Text Criteria
• Authors: Sehyun Kwon, Jaeseung Park, Minkyu Kim, Jaewoong Cho, Ernest K. Ryu, Kangwook Lee
• Abstract summary: We present a new method for performing image clustering based on user-specified text criteria. We call our method Image Clustering Conditioned on Text Criteria (IC|TC) IC|TC requires a minimal and practical degree of human intervention and grants the user significant control over the clustering results in return.
• Score: 14.704110575570166
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Classical clustering methods do not provide users with direct control of the clustering results, and the clustering results may not be consistent with the relevant criterion that a user has in mind. In this work, we present a new methodology for performing image clustering based on user-specified text criteria by leveraging modern vision-language models and large language models. We call our method Image Clustering Conditioned on Text Criteria (IC|TC), and it represents a different paradigm of image clustering. IC|TC requires a minimal and practical degree of human intervention and grants the user significant control over the clustering results in return. Our experiments show that IC|TC can effectively cluster images with various criteria, such as human action, physical location, or the person's mood, while significantly outperforming baselines.

Related papers

• Organizing Unstructured Image Collections using Natural Language [37.16101036513514]
  We introduce the task Semantic Multiple Clustering (SMC) that aims to automatically discover clustering criteria from large image collections. Our framework, Text Driven Semantic Multiple Clustering (TeDeSC), uses text as a proxy to concurrently reason over large image collections. We apply TeDeSC to various applications, such as discovering biases and analyzing social media image popularity.
  arXiv  Detail & Related papers  (2024-10-07T17:21:46Z)
• Text-Guided Alternative Image Clustering [11.103514372355088]
  This work explores the potential of large vision-language models to facilitate alternative image clustering. We propose Text-Guided Alternative Image Consensus Clustering (TGAICC), a novel approach that leverages user-specified interests via prompts. TGAICC outperforms image- and text-based baselines on four alternative image clustering benchmark datasets.
  arXiv  Detail & Related papers  (2024-06-07T08:37:57Z)
• Image Clustering with External Guidance [33.664812922814754]
  The core of clustering is incorporating prior knowledge to construct supervision signals. We propose leveraging external knowledge as a new supervision signal to guide clustering, even though it seems irrelevant to the given data.
  arXiv  Detail & Related papers  (2023-10-18T14:20:55Z)
• CoC-GAN: Employing Context Cluster for Unveiling a New Pathway in Image Generation [12.211795836214112]
  We propose a unique image generation process premised on the perspective of converting images into a set of point clouds. Our methodology leverages simple clustering methods named Context Clustering (CoC) to generate images from unordered point sets. We introduce this model with the novel structure as the Context Clustering Generative Adversarial Network (CoC-GAN)
  arXiv  Detail & Related papers  (2023-08-23T01:19:58Z)
• On Mitigating Hard Clusters for Face Clustering [48.39472979642971]
  Face clustering is a promising way to scale up face recognition systems using large-scale unlabeled face images. We introduce two novel modules, Neighborhood-Diffusion-based Density (NDDe) and Transition-Probability-based Distance (TPDi) Our experiments on multiple benchmarks show that each module contributes to the final performance of our method.
  arXiv  Detail & Related papers  (2022-07-25T03:55:15Z)
• Using Representation Expressiveness and Learnability to Evaluate Self-Supervised Learning Methods [61.49061000562676]
  We introduce Cluster Learnability (CL) to assess learnability. CL is measured in terms of the performance of a KNN trained to predict labels obtained by clustering the representations with K-means. We find that CL better correlates with in-distribution model performance than other competing recent evaluation schemes.
  arXiv  Detail & Related papers  (2022-06-02T19:05:13Z)
• Clustering by Maximizing Mutual Information Across Views [62.21716612888669]
  We propose a novel framework for image clustering that incorporates joint representation learning and clustering. Our method significantly outperforms state-of-the-art single-stage clustering methods across a variety of image datasets.
  arXiv  Detail & Related papers  (2021-07-24T15:36:49Z)
• You Never Cluster Alone [150.94921340034688]
  We extend the mainstream contrastive learning paradigm to a cluster-level scheme, where all the data subjected to the same cluster contribute to a unified representation. We define a set of categorical variables as clustering assignment confidence, which links the instance-level learning track with the cluster-level one. By reparametrizing the assignment variables, TCC is trained end-to-end, requiring no alternating steps.
  arXiv  Detail & Related papers  (2021-06-03T14:59:59Z)
• Graph Contrastive Clustering [131.67881457114316]
  We propose a novel graph contrastive learning framework, which is then applied to the clustering task and we come up with the Graph Constrastive Clustering(GCC) method. Specifically, on the one hand, the graph Laplacian based contrastive loss is proposed to learn more discriminative and clustering-friendly features. On the other hand, a novel graph-based contrastive learning strategy is proposed to learn more compact clustering assignments.
  arXiv  Detail & Related papers  (2021-04-03T15:32:49Z)
• Revisiting Contrastive Learning for Few-Shot Classification [74.78397993160583]
  Instance discrimination based contrastive learning has emerged as a leading approach for self-supervised learning of visual representations. We show how one can incorporate supervision in the instance discrimination based contrastive self-supervised learning framework to learn representations that generalize better to novel tasks. We propose a novel model selection algorithm that can be used in conjunction with a universal embedding trained using CIDS to outperform state-of-the-art algorithms on the challenging Meta-Dataset benchmark.
  arXiv  Detail & Related papers  (2021-01-26T19:58:08Z)
• Consensus Clustering With Unsupervised Representation Learning [4.164845768197489]
  We study the clustering ability of Bootstrap Your Own Latent (BYOL) and observe that features learnt using BYOL may not be optimal for clustering. We propose a novel consensus clustering based loss function, and train BYOL with the proposed loss in an end-to-end way that improves the clustering ability and outperforms similar clustering based methods.
  arXiv  Detail & Related papers  (2020-10-03T01:16:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.