Classifier Crafting: Turn Your ConvNet into a Zero-Shot Learner!

• URL: http://arxiv.org/abs/2103.11112v1
• Date: Sat, 20 Mar 2021 06:26:29 GMT
• Title: Classifier Crafting: Turn Your ConvNet into a Zero-Shot Learner!
• Authors: Jacopo Cavazza
• Abstract summary: We tackle Zero-shot learning (ZSL) by casting a convolutional neural network into a zero-shot learner. We learn a data-driven and ZSL-tailored feature representation on seen classes only to match these fixed classification rules. We can perform ZSL inference by augmenting the pool of classification rules at test time while keeping the very same representation we learnt.
• Score: 5.3556221126231085
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In Zero-shot learning (ZSL), we classify unseen categories using textual descriptions about their expected appearance when observed (class embeddings) and a disjoint pool of seen classes, for which annotated visual data are accessible. We tackle ZSL by casting a "vanilla" convolutional neural network (e.g. AlexNet, ResNet-101, DenseNet-201 or DarkNet-53) into a zero-shot learner. We do so by crafting the softmax classifier: we freeze its weights using fixed seen classification rules, either semantic (seen class embeddings) or visual (seen class prototypes). Then, we learn a data-driven and ZSL-tailored feature representation on seen classes only to match these fixed classification rules. Given that the latter seamlessly generalize towards unseen classes, while requiring not actual unseen data to be computed, we can perform ZSL inference by augmenting the pool of classification rules at test time while keeping the very same representation we learnt: nowhere re-training or fine-tuning on unseen data is performed. The combination of semantic and visual crafting (by simply averaging softmax scores) improves prior state-of-the-art methods in benchmark datasets for standard, inductive ZSL. After rebalancing predictions to better handle the joint inference over seen and unseen classes, we outperform prior generalized, inductive ZSL methods as well. Also, we gain interpretability at no additional cost, by using neural attention methods (e.g., grad-CAM) as they are. Code will be made publicly available.

Related papers

• A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation [121.0693322732454]
  Contrastive Language-Image Pretraining (CLIP) has gained popularity for its remarkable zero-shot capacity. Recent research has focused on developing efficient fine-tuning methods to enhance CLIP's performance in downstream tasks. We revisit a classical algorithm, Gaussian Discriminant Analysis (GDA), and apply it to the downstream classification of CLIP.
  arXiv  Detail & Related papers  (2024-02-06T15:45:27Z)
• Data-Free Generalized Zero-Shot Learning [45.86614536578522]
  We propose a generic framework for data-free zero-shot learning (DFZSL) Our framework has been evaluated on five commonly used benchmarks for generalized ZSL, as well as 11 benchmarks for the base-to-new ZSL.
  arXiv  Detail & Related papers  (2024-01-28T13:26:47Z)
• Self-Supervised Class Incremental Learning [51.62542103481908]
  Existing Class Incremental Learning (CIL) methods are based on a supervised classification framework sensitive to data labels. When updating them based on the new class data, they suffer from catastrophic forgetting: the model cannot discern old class data clearly from the new. In this paper, we explore the performance of Self-Supervised representation learning in Class Incremental Learning (SSCIL) for the first time.
  arXiv  Detail & Related papers  (2021-11-18T06:58:19Z)
• Learning without Seeing nor Knowing: Towards Open Zero-Shot Learning [27.283748476678117]
  In Generalized Zero-Shot Learning (GZSL) unseen categories can be predicted by leveraging their class embeddings. We propose Open Zero-Shot Learning (OZSL) to extend GZSL towards the open-world settings. We formalize OZSL as the problem of recognizing seen and unseen classes while also rejecting instances from unknown categories.
  arXiv  Detail & Related papers  (2021-03-23T10:30:50Z)
• Counterfactual Zero-Shot and Open-Set Visual Recognition [95.43275761833804]
  We present a novel counterfactual framework for both Zero-Shot Learning (ZSL) and Open-Set Recognition (OSR) Our idea stems from the observation that the generated samples for unseen-classes are often out of the true distribution. We demonstrate that our framework effectively mitigates the seen/unseen imbalance and hence significantly improves the overall performance.
  arXiv  Detail & Related papers  (2021-03-01T10:20:04Z)
• CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognition [52.66360172784038]
  We propose a clustering-based model, which considers all training samples at once, instead of optimizing for each instance individually. We call the proposed method CLASTER and observe that it consistently improves over the state-of-the-art in all standard datasets.
  arXiv  Detail & Related papers  (2021-01-18T12:46:24Z)
• Attribute Propagation Network for Graph Zero-shot Learning [57.68486382473194]
  We introduce the attribute propagation network (APNet), which is composed of 1) a graph propagation model generating attribute vector for each class and 2) a parameterized nearest neighbor (NN) classifier. APNet achieves either compelling performance or new state-of-the-art results in experiments with two zero-shot learning settings and five benchmark datasets.
  arXiv  Detail & Related papers  (2020-09-24T16:53:40Z)
• Information Bottleneck Constrained Latent Bidirectional Embedding for Zero-Shot Learning [59.58381904522967]
  We propose a novel embedding based generative model with a tight visual-semantic coupling constraint. We learn a unified latent space that calibrates the embedded parametric distributions of both visual and semantic spaces. Our method can be easily extended to transductive ZSL setting by generating labels for unseen images.
  arXiv  Detail & Related papers  (2020-09-16T03:54:12Z)
• Leveraging Seen and Unseen Semantic Relationships for Generative Zero-Shot Learning [14.277015352910674]
  We propose a generative model that explicitly performs knowledge transfer by incorporating a novel Semantic Regularized Loss (SR-Loss) Experiments on seven benchmark datasets demonstrate the superiority of the LsrGAN compared to previous state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-07-19T01:25:53Z)
• Generative Adversarial Zero-shot Learning via Knowledge Graphs [32.42721467499858]
  We introduce a new generative ZSL method named KG-GAN by incorporating rich semantics in a knowledge graph (KG) into GANs. Specifically, we build upon Graph Neural Networks and encode KG from two views: class view and attribute view. With well-learned semantic embeddings for each node (representing a visual category), we leverage GANs to synthesize compelling visual features for unseen classes.
  arXiv  Detail & Related papers  (2020-04-07T03:55:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.