Classifier-guided CLIP Distillation for Unsupervised Multi-label Classification

• URL: http://arxiv.org/abs/2503.16873v1
• Date: Fri, 21 Mar 2025 06:12:14 GMT
• Title: Classifier-guided CLIP Distillation for Unsupervised Multi-label Classification
• Authors: Dongseob Kim, Hyunjung Shim,
• Abstract summary: Multi-label classification is crucial for comprehensive image understanding.<n>Despite CLIP's proficiency, it suffers from view-dependent predictions and inherent bias, limiting its effectiveness.<n>We propose a novel method that addresses these issues by leveraging multiple views near target objects.
• Score: 16.0058187276343
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Multi-label classification is crucial for comprehensive image understanding, yet acquiring accurate annotations is challenging and costly. To address this, a recent study suggests exploiting unsupervised multi-label classification leveraging CLIP, a powerful vision-language model. Despite CLIP's proficiency, it suffers from view-dependent predictions and inherent bias, limiting its effectiveness. We propose a novel method that addresses these issues by leveraging multiple views near target objects, guided by Class Activation Mapping (CAM) of the classifier, and debiasing pseudo-labels derived from CLIP predictions. Our Classifier-guided CLIP Distillation (CCD) enables selecting multiple local views without extra labels and debiasing predictions to enhance classification performance. Experimental results validate our method's superiority over existing techniques across diverse datasets. The code is available at https://github.com/k0u-id/CCD.

Related papers

• Collaborative Feature-Logits Contrastive Learning for Open-Set Semi-Supervised Object Detection [75.02249869573994]
  In open-set scenarios, the unlabeled dataset contains both in-distribution (ID) classes and out-of-distribution (OOD) classes.<n>Applying semi-supervised detectors in such settings can lead to misclassifying OOD class as ID classes.<n>We propose a simple yet effective method, termed Collaborative Feature-Logits Detector (CFL-Detector)
  arXiv  Detail & Related papers  (2024-11-20T02:57:35Z)
• Multi-label Cluster Discrimination for Visual Representation Learning [27.552024985952166]
  We propose a novel Multi-Label Cluster Discrimination method named MLCD to enhance representation learning. Our method achieves state-of-the-art performance on multiple downstream tasks including linear probe, zero-shot classification, and image-text retrieval.
  arXiv  Detail & Related papers  (2024-07-24T14:54:16Z)
• TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP Without Training [29.431698321195814]
  Contrastive Language-Image Pre-training (CLIP) has demonstrated impressive capabilities in open-vocabulary classification. CLIP shows poor performance on multi-label datasets because the global feature tends to be dominated by the most prominent class. We propose a local-to-global framework to obtain image tags.
  arXiv  Detail & Related papers  (2023-12-20T08:15:40Z)
• Virtual Category Learning: A Semi-Supervised Learning Method for Dense Prediction with Extremely Limited Labels [63.16824565919966]
  This paper proposes to use confusing samples proactively without label correction. A Virtual Category (VC) is assigned to each confusing sample in such a way that it can safely contribute to the model optimisation. Our intriguing findings highlight the usage of VC learning in dense vision tasks.
  arXiv  Detail & Related papers  (2023-12-02T16:23:52Z)
• CDUL: CLIP-Driven Unsupervised Learning for Multi-Label Image Classification [23.392746466420128]
  This paper presents a CLIP-based unsupervised learning method for annotation-free multi-label image classification. We take full advantage of the powerful CLIP model and propose a novel approach to extend CLIP for multi-label predictions based on global-local image-text similarity aggregation. Our method outperforms state-of-the-art unsupervised methods on MS-COCO, PASCAL VOC 2007, PASCAL VOC 2012, and NUS datasets.
  arXiv  Detail & Related papers  (2023-07-31T13:12:02Z)
• CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Grounding [86.79903269137971]
  Unsupervised visual grounding has been developed to locate regions using pseudo-labels. We propose CLIP-VG, a novel method that can conduct self-paced curriculum adapting of CLIP with pseudo-language labels. Our method outperforms the current state-of-the-art unsupervised method by a significant margin on RefCOCO/+/g datasets.
  arXiv  Detail & Related papers  (2023-05-15T14:42:02Z)
• Transductive CLIP with Class-Conditional Contrastive Learning [68.51078382124331]
  We propose Transductive CLIP, a novel framework for learning a classification network with noisy labels from scratch. A class-conditional contrastive learning mechanism is proposed to mitigate the reliance on pseudo labels. ensemble labels is adopted as a pseudo label updating strategy to stabilize the training of deep neural networks with noisy labels.
  arXiv  Detail & Related papers  (2022-06-13T14:04:57Z)
• Fine-Grained Visual Classification using Self Assessment Classifier [12.596520707449027]
  Extracting discriminative features plays a crucial role in the fine-grained visual classification task. In this paper, we introduce a Self Assessment, which simultaneously leverages the representation of the image and top-k prediction classes. We show that our method achieves new state-of-the-art results on CUB200-2011, Stanford Dog, and FGVC Aircraft datasets.
  arXiv  Detail & Related papers  (2022-05-21T07:41:27Z)
• DenseCLIP: Extract Free Dense Labels from CLIP [130.3830819077699]
  Contrastive Language-Image Pre-training (CLIP) has made a remarkable breakthrough in open-vocabulary zero-shot image recognition. DenseCLIP+ surpasses SOTA transductive zero-shot semantic segmentation methods by large margins. Our finding suggests that DenseCLIP can serve as a new reliable source of supervision for dense prediction tasks.
  arXiv  Detail & Related papers  (2021-12-02T09:23:01Z)
• Binary Classification from Multiple Unlabeled Datasets via Surrogate Set Classification [94.55805516167369]
  We propose a new approach for binary classification from m U-sets for $mge2$. Our key idea is to consider an auxiliary classification task called surrogate set classification (SSC)
  arXiv  Detail & Related papers  (2021-02-01T07:36:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.