Language-Assisted Image Clustering Guided by Discriminative Relational Signals and Adaptive Semantic Centers

• URL: http://arxiv.org/abs/2603.24275v1
• Date: Wed, 25 Mar 2026 13:08:14 GMT
• Title: Language-Assisted Image Clustering Guided by Discriminative Relational Signals and Adaptive Semantic Centers
• Abstract summary: We propose a new Language-Assisted Image Clustering (LAIC) framework with two complementary components.<n>We exploit cross-language relations to produce more discriminative self-supervision signals for clustering.<n>We learn category-wise continuous semantic centers via prompt learning to produce the final clustering assignments.
• Score: 65.70472330370735
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Language-Assisted Image Clustering (LAIC) augments the input images with additional texts with the help of vision-language models (VLMs) to promote clustering performance. Despite recent progress, existing LAIC methods often overlook two issues: (i) textual features constructed for each image are highly similar, leading to weak inter-class discriminability; (ii) the clustering step is restricted to pre-built image-text alignments, limiting the potential for better utilization of the text modality. To address these issues, we propose a new LAIC framework with two complementary components. First, we exploit cross-modal relations to produce more discriminative self-supervision signals for clustering, as it compatible with most VLMs training mechanisms. Second, we learn category-wise continuous semantic centers via prompt learning to produce the final clustering assignments. Extensive experiments on eight benchmark datasets demonstrate that our method achieves an average improvement of 2.6% over state-of-the-art methods, and the learned semantic centers exhibit strong interpretability. Code is available in the supplementary material.

Related papers

• Hierarchical Semantic Alignment for Image Clustering [59.277605709780524]
  We propose a hierarChical semAntic alignmEnt method for image clustering, dubbed CAE, which improves cluster- ing performance in a training-free manner.<n>We first select relevant nouns from WordNet and descriptions from caption datasets to construct a semantic space aligned with image features.<n>Then, we align image features with selected nouns and captions via optimal transport to obtain a more discriminative semantic space.
  arXiv  Detail & Related papers  (2025-11-30T14:14:51Z)
• Synthetic Captions for Open-Vocabulary Zero-Shot Segmentation [6.004292247258359]
  We show how to densely align images with synthetic descriptions generated by generative vision-language models.<n>Our approach outperforms prior work on standard zero-shot open-vocabulary segmentation benchmarks/datasets.
  arXiv  Detail & Related papers  (2025-09-15T12:26:47Z)
• Gradient-Attention Guided Dual-Masking Synergetic Framework for Robust Text-based Person Retrieval [15.126709823382539]
  This work advances Contrastive Language-Image Pre-training (CLIP) for person representation learning.<n>We develop a noise-resistant data construction pipeline that leverages the in-context learning capabilities of MLLMs.<n>We introduce the GA-DMS framework, which improves cross-modal alignment by adaptively masking noisy textual tokens.
  arXiv  Detail & Related papers  (2025-09-11T03:06:22Z)
• VladVA: Discriminative Fine-tuning of LVLMs [67.14293827774827]
  Contrastively-trained Vision-Language Models (VLMs) like CLIP have become the de facto approach for discriminative vision-language representation learning.<n>We propose to combine "the best of both worlds": a new training approach for discriminative fine-tuning of LVLMs.
  arXiv  Detail & Related papers  (2024-12-05T17:54:27Z)
• Dual-Level Cross-Modal Contrastive Clustering [4.083185193413678]
  We propose a novel image clustering framwork, named Dual-level Cross-Modal Contrastive Clustering (DXMC) external textual information is introduced for constructing a semantic space which is adopted to generate image-text pairs. The image-text pairs are respectively sent to pre-trained image and text encoder to obtain image and text embeddings which subsquently are fed into four well-designed networks.
  arXiv  Detail & Related papers  (2024-09-06T18:49:45Z)
• Dual Advancement of Representation Learning and Clustering for Sparse and Noisy Images [14.836487514037994]
  Sparse and noisy images (SNIs) pose significant challenges for effective representation learning and clustering. We propose Dual Advancement of Representation Learning and Clustering (DARLC) to enhance the representations derived from masked image modeling. Our framework offers a comprehensive approach that improves the learning of representations by enhancing their local perceptibility, distinctiveness, and the understanding of relational semantics.
  arXiv  Detail & Related papers  (2024-09-03T10:52:27Z)
• Multi-Grained Cross-modal Alignment for Learning Open-vocabulary Semantic Segmentation from Text Supervision [23.931443799102663]
  We introduce a Multi-Grained Cross-modal Alignment (MGCA) framework to bridge the granularity gap without any dense annotations. Specifically, MGCA constructs pseudo multi-granular semantic correspondences upon image-text pairs. Our method achieves significant advancements over state-of-the-art methods, demonstrating its effectiveness and efficiency.
  arXiv  Detail & Related papers  (2024-03-06T13:43:36Z)
• Non-Contrastive Learning Meets Language-Image Pre-Training [145.6671909437841]
  We study the validity of non-contrastive language-image pre-training (nCLIP) We introduce xCLIP, a multi-tasking framework combining CLIP and nCLIP, and show that nCLIP aids CLIP in enhancing feature semantics.
  arXiv  Detail & Related papers  (2022-10-17T17:57:46Z)
• MaPLe: Multi-modal Prompt Learning [54.96069171726668]
  We propose Multi-modal Prompt Learning (MaPLe) for both vision and language branches to improve alignment between the vision and language representations. Compared with the state-of-the-art method Co-CoOp, MaPLe exhibits favorable performance and achieves an absolute gain of 3.45% on novel classes.
  arXiv  Detail & Related papers  (2022-10-06T17:59:56Z)
• COTS: Collaborative Two-Stream Vision-Language Pre-Training Model for Cross-Modal Retrieval [59.15034487974549]
  We propose a novel COllaborative Two-Stream vision-language pretraining model termed COTS for image-text retrieval. Our COTS achieves the highest performance among all two-stream methods and comparable performance with 10,800X faster in inference. Importantly, our COTS is also applicable to text-to-video retrieval, yielding new state-ofthe-art on the widely-used MSR-VTT dataset.
  arXiv  Detail & Related papers  (2022-04-15T12:34:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.