Label Propagation for Zero-shot Classification with Vision-Language Models

• URL: http://arxiv.org/abs/2404.04072v1
• Date: Fri, 5 Apr 2024 12:58:07 GMT
• Title: Label Propagation for Zero-shot Classification with Vision-Language Models
• Authors: Vladan Stojnić, Yannis Kalantidis, Giorgos Tolias,
• Abstract summary: In this paper, we tackle the case of zero-shot classification in the presence of unlabeled data. We introduce ZLaP, a method based on label propagation (LP) that utilizes geodesic distances for classification. We perform extensive experiments to evaluate the effectiveness of our method on 14 common datasets and show that ZLaP outperforms the latest related works.
• Score: 17.50253820510074
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language Models (VLMs) have demonstrated impressive performance on zero-shot classification, i.e. classification when provided merely with a list of class names. In this paper, we tackle the case of zero-shot classification in the presence of unlabeled data. We leverage the graph structure of the unlabeled data and introduce ZLaP, a method based on label propagation (LP) that utilizes geodesic distances for classification. We tailor LP to graphs containing both text and image features and further propose an efficient method for performing inductive inference based on a dual solution and a sparsification step. We perform extensive experiments to evaluate the effectiveness of our method on 14 common datasets and show that ZLaP outperforms the latest related works. Code: https://github.com/vladan-stojnic/ZLaP

Related papers

• Label Set Optimization via Activation Distribution Kurtosis for Zero-shot Classification with Generative Models [10.699636123243138]
  In-context learning (ICL) performance is sensitive to the prompt design, yet the impact of class label options in zero-shot classification has been largely overlooked. This study presents the first comprehensive empirical study investigating how label option influences zero-shot ICL classification performance.
  arXiv  Detail & Related papers  (2024-10-24T22:59:23Z)
• Posterior Label Smoothing for Node Classification [2.737276507021477]
  We propose a simple yet effective label smoothing for the transductive node classification task. We design the soft label to encapsulate the local context of the target node through the neighborhood label distribution. In the following analysis, we find that incorporating global label statistics in posterior computation is the key to the success of label smoothing.
  arXiv  Detail & Related papers  (2024-06-01T11:59:49Z)
• VLM-CPL: Consensus Pseudo Labels from Vision-Language Models for Human Annotation-Free Pathological Image Classification [23.08368823707528]
  We present a novel human annotation-free method for pathology image classification by leveraging pre-trained Vision-Language Models (VLMs) We introduce VLM-CPL, a novel approach based on consensus pseudo labels that integrates two noisy label filtering techniques with a semi-supervised learning strategy. Experimental results showed that our method obtained an accuracy of 87.1% and 95.1% on the HPH and LC25K datasets, respectively.
  arXiv  Detail & Related papers  (2024-03-23T13:24:30Z)
• LaFTer: Label-Free Tuning of Zero-shot Classifier using Language and Unlabeled Image Collections [30.875186985461063]
  Large-scale pre-trained Vision and Language (VL) models have set a new state-of-the-art (SOTA) in zero-shot visual classification. We show, for the first time, how to reduce this gap without any labels and without any paired VL data.
  arXiv  Detail & Related papers  (2023-05-29T17:56:35Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• Binary Classification with Positive Labeling Sources [71.37692084951355]
  We propose WEAPO, a simple yet competitive WS method for producing training labels without negative labeling sources. We show WEAPO achieves the highest averaged performance on 10 benchmark datasets.
  arXiv  Detail & Related papers  (2022-08-02T19:32:08Z)
• Self-Adaptive Label Augmentation for Semi-supervised Few-shot Classification [121.63992191386502]
  Few-shot classification aims to learn a model that can generalize well to new tasks when only a few labeled samples are available. We propose a semi-supervised few-shot classification method that assigns an appropriate label to each unlabeled sample by a manually defined metric. A major novelty of SALA is the task-adaptive metric, which can learn the metric adaptively for different tasks in an end-to-end fashion.
  arXiv  Detail & Related papers  (2022-06-16T13:14:03Z)
• Multi-label Classification with Partial Annotations using Class-aware Selective Loss [14.3159150577502]
  Large-scale multi-label classification datasets are commonly partially annotated. We analyze the partial labeling problem, then propose a solution based on two key ideas. With our novel approach, we achieve state-of-the-art results on OpenImages dataset.
  arXiv  Detail & Related papers  (2021-10-21T08:10:55Z)
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption [72.35532598131176]
  We propose SCARF, a technique for contrastive learning, where views are formed by corrupting a random subset of features. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders.
  arXiv  Detail & Related papers  (2021-06-29T08:08:33Z)
• SLADE: A Self-Training Framework For Distance Metric Learning [75.54078592084217]
  We present a self-training framework, SLADE, to improve retrieval performance by leveraging additional unlabeled data. We first train a teacher model on the labeled data and use it to generate pseudo labels for the unlabeled data. We then train a student model on both labels and pseudo labels to generate final feature embeddings.
  arXiv  Detail & Related papers  (2020-11-20T08:26:10Z)
• An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels [49.036212158261215]
  Large-scale Multi-label Text Classification (LMTC) has a wide range of Natural Language Processing (NLP) applications. Current state-of-the-art LMTC models employ Label-Wise Attention Networks (LWANs) We show that hierarchical methods based on Probabilistic Label Trees (PLTs) outperform LWANs. We propose a new state-of-the-art method which combines BERT with LWANs.
  arXiv  Detail & Related papers  (2020-10-04T18:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.