LLM meets Vision-Language Models for Zero-Shot One-Class Classification

• URL: http://arxiv.org/abs/2404.00675v3
• Date: Mon, 27 May 2024 08:53:15 GMT
• Title: LLM meets Vision-Language Models for Zero-Shot One-Class Classification
• Authors: Yassir Bendou, Giulia Lioi, Bastien Pasdeloup, Lukas Mauch, Ghouthi Boukli Hacene, Fabien Cardinaux, Vincent Gripon,
• Abstract summary: We consider the problem of zero-shot one-class visual classification. We propose a two-step solution that first queries large language models for visually confusing objects. We are the first to demonstrate the ability to discriminate a single category from other semantically related ones using only its label.
• Score: 4.094697851983375
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We consider the problem of zero-shot one-class visual classification, extending traditional one-class classification to scenarios where only the label of the target class is available. This method aims to discriminate between positive and negative query samples without requiring examples from the target class. We propose a two-step solution that first queries large language models for visually confusing objects and then relies on vision-language pre-trained models (e.g., CLIP) to perform classification. By adapting large-scale vision benchmarks, we demonstrate the ability of the proposed method to outperform adapted off-the-shelf alternatives in this setting. Namely, we propose a realistic benchmark where negative query samples are drawn from the same original dataset as positive ones, including a granularity-controlled version of iNaturalist, where negative samples are at a fixed distance in the taxonomy tree from the positive ones. To our knowledge, we are the first to demonstrate the ability to discriminate a single category from other semantically related ones using only its label.

Related papers

• Negative Prototypes Guided Contrastive Learning for WSOD [8.102080369924911]
  Weakly Supervised Object Detection (WSOD) with only image-level annotation has recently attracted wide attention. We propose the Negative Prototypes Guided Contrastive learning architecture. Our proposed method achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-06-04T08:16:26Z)
• Liberating Seen Classes: Boosting Few-Shot and Zero-Shot Text Classification via Anchor Generation and Classification Reframing [38.84431954053434]
  Few-shot and zero-shot text classification aim to recognize samples from novel classes with limited labeled samples or no labeled samples at all. We propose a simple and effective strategy for few-shot and zero-shot text classification.
  arXiv  Detail & Related papers  (2024-05-06T15:38:32Z)
• Embracing Diversity: Interpretable Zero-shot classification beyond one vector per class [16.101460010750458]
  We argue that to represent diversity within a class, zero-shot classification should move beyond a single vector. We propose a method to encode and account for diversity within a class using inferred attributes, still in the zero-shot setting without retraining. We find our method consistently outperforms standard zero-shot classification over a large suite of datasets.
  arXiv  Detail & Related papers  (2024-04-25T16:29:06Z)
• Towards Realistic Zero-Shot Classification via Self Structural Semantic Alignment [53.2701026843921]
  Large-scale pre-trained Vision Language Models (VLMs) have proven effective for zero-shot classification. In this paper, we aim at a more challenging setting, Realistic Zero-Shot Classification, which assumes no annotation but instead a broad vocabulary. We propose the Self Structural Semantic Alignment (S3A) framework, which extracts structural semantic information from unlabeled data while simultaneously self-learning.
  arXiv  Detail & Related papers  (2023-08-24T17:56:46Z)
• Reflection Invariance Learning for Few-shot Semantic Segmentation [53.20466630330429]
  Few-shot semantic segmentation (FSS) aims to segment objects of unseen classes in query images with only a few annotated support images. This paper proposes a fresh few-shot segmentation framework to mine the reflection invariance in a multi-view matching manner. Experiments on both PASCAL-$5textiti$ and COCO-$20textiti$ datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-06-01T15:14:58Z)
• Learning Classifiers of Prototypes and Reciprocal Points for Universal Domain Adaptation [79.62038105814658]
  Universal Domain aims to transfer the knowledge between datasets by handling two shifts: domain-shift and categoryshift. Main challenge is correctly distinguishing the unknown target samples while adapting the distribution of known class knowledge from source to target. Most existing methods approach this problem by first training the target adapted known and then relying on the single threshold to distinguish unknown target samples.
  arXiv  Detail & Related papers  (2022-12-16T09:01:57Z)
• A Gating Model for Bias Calibration in Generalized Zero-shot Learning [18.32369721322249]
  Generalized zero-shot learning (GZSL) aims at training a model that can generalize to unseen class data by only using auxiliary information. One of the main challenges in GZSL is a biased model prediction toward seen classes caused by overfitting on only available seen class data during training. We propose a two-stream autoencoder-based gating model for GZSL.
  arXiv  Detail & Related papers  (2022-03-08T16:41:06Z)
• CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognition [52.66360172784038]
  We propose a clustering-based model, which considers all training samples at once, instead of optimizing for each instance individually. We call the proposed method CLASTER and observe that it consistently improves over the state-of-the-art in all standard datasets.
  arXiv  Detail & Related papers  (2021-01-18T12:46:24Z)
• Learning and Evaluating Representations for Deep One-class Classification [59.095144932794646]
  We present a two-stage framework for deep one-class classification. We first learn self-supervised representations from one-class data, and then build one-class classifiers on learned representations. In experiments, we demonstrate state-of-the-art performance on visual domain one-class classification benchmarks.
  arXiv  Detail & Related papers  (2020-11-04T23:33:41Z)
• Meta Learning for Few-Shot One-class Classification [0.0]
  We formulate the learning of meaningful features for one-class classification as a meta-learning problem. To learn these representations, we require only multiclass data from similar tasks. We validate our approach by adapting few-shot classification datasets to the few-shot one-class classification scenario.
  arXiv  Detail & Related papers  (2020-09-11T11:35:28Z)
• Few-shot Classification via Adaptive Attention [93.06105498633492]
  We propose a novel few-shot learning method via optimizing and fast adapting the query sample representation based on very few reference samples. As demonstrated experimentally, the proposed model achieves state-of-the-art classification results on various benchmark few-shot classification and fine-grained recognition datasets.
  arXiv  Detail & Related papers  (2020-08-06T05:52:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.