A Multi-class Approach -- Building a Visual Classifier based on Textual Descriptions using Zero-Shot Learning

• URL: http://arxiv.org/abs/2011.09236v1
• Date: Wed, 18 Nov 2020 12:06:55 GMT
• Title: A Multi-class Approach -- Building a Visual Classifier based on Textual Descriptions using Zero-Shot Learning
• Authors: Preeti Jagdish Sajjan and Frank G. Glavin
• Abstract summary: We overcome the two main hurdles of Machine Learning, i.e. scarcity of data and constrained prediction of the classification model. We train a classifier by mapping labelled images to their textual description instead of training it for specific classes.
• Score: 0.34265828682659694
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine Learning (ML) techniques for image classification routinely require many labelled images for training the model and while testing, we ought to use images belonging to the same domain as those used for training. In this paper, we overcome the two main hurdles of ML, i.e. scarcity of data and constrained prediction of the classification model. We do this by introducing a visual classifier which uses a concept of transfer learning, namely Zero-Shot Learning (ZSL), and standard Natural Language Processing techniques. We train a classifier by mapping labelled images to their textual description instead of training it for specific classes. Transfer learning involves transferring knowledge across domains that are similar. ZSL intelligently applies the knowledge learned while training for future recognition tasks. ZSL differentiates classes as two types: seen and unseen classes. Seen classes are the classes upon which we have trained our model and unseen classes are the classes upon which we test our model. The examples from unseen classes have not been encountered in the training phase. Earlier research in this domain focused on developing a binary classifier but, in this paper, we present a multi-class classifier with a Zero-Shot Learning approach.

Related papers

• Premonition: Using Generative Models to Preempt Future Data Changes in Continual Learning [63.850451635362425]
  Continual learning requires a model to adapt to ongoing changes in the data distribution. We show that the combination of a large language model and an image generation model can similarly provide useful premonitions. We find that the backbone of our pre-trained networks can learn representations useful for the downstream continual learning problem.
  arXiv  Detail & Related papers  (2024-03-12T06:29:54Z)
• Deep Semantic-Visual Alignment for Zero-Shot Remote Sensing Image Scene Classification [26.340737217001497]
  Zero-shot learning (ZSL) allows for identifying novel classes that are not seen during training. Previous ZSL models mainly depend on manually-labeled attributes or word embeddings extracted from language models to transfer knowledge from seen classes to novel classes. We propose to collect visually detectable attributes automatically. We predict attributes for each class by depicting the semantic-visual similarity between attributes and images.
  arXiv  Detail & Related papers  (2024-02-03T09:18:49Z)
• Image-free Classifier Injection for Zero-Shot Classification [72.66409483088995]
  Zero-shot learning models achieve remarkable results on image classification for samples from classes that were not seen during training. We aim to equip pre-trained models with zero-shot classification capabilities without the use of image data. We achieve this with our proposed Image-free Injection with Semantics (ICIS)
  arXiv  Detail & Related papers  (2023-08-21T09:56:48Z)
• DiRaC-I: Identifying Diverse and Rare Training Classes for Zero-Shot Learning [6.75714270653184]
  It is intuitive that intelligently selecting the training classes from a dataset for Zero-Shot Learning (ZSL) can improve the performance of existing ZSL methods. We propose a framework called Diverse and Rare Class Identifier (DiRaC-I) which, given an attribute-based dataset, can intelligently yield the most suitable "seen classes" for training ZSL models. Our results demonstrate DiRaC-I helps ZSL models to achieve significant classification accuracy improvements.
  arXiv  Detail & Related papers  (2022-12-31T16:05:09Z)
• Learning Semantic Ambiguities for Zero-Shot Learning [0.0]
  We propose a regularization method that can be applied to any conditional generative-based ZSL method. It learns to synthesize discriminative features for possible semantic description that are not available at training time, that is the unseen ones. The approach is evaluated for ZSL and GZSL on four datasets commonly used in the literature.
  arXiv  Detail & Related papers  (2022-01-05T21:08:29Z)
• Rectifying the Shortcut Learning of Background: Shared Object Concentration for Few-Shot Image Recognition [101.59989523028264]
  Few-Shot image classification aims to utilize pretrained knowledge learned from a large-scale dataset to tackle a series of downstream classification tasks. We propose COSOC, a novel Few-Shot Learning framework, to automatically figure out foreground objects at both pretraining and evaluation stage.
  arXiv  Detail & Related papers  (2021-07-16T07:46:41Z)
• Aligning Visual Prototypes with BERT Embeddings for Few-Shot Learning [48.583388368897126]
  Few-shot learning is the task of learning to recognize previously unseen categories of images. We propose a method that takes into account the names of the image classes.
  arXiv  Detail & Related papers  (2021-05-21T08:08:28Z)
• OntoZSL: Ontology-enhanced Zero-shot Learning [19.87808305218359]
  Key to implementing Zero-shot Learning (ZSL) is to leverage the prior knowledge of classes which builds the semantic relationship between classes. In this paper, we explore richer and more competitive prior knowledge to model the inter-class relationship for ZSL. To address the data imbalance between seen classes and unseen classes, we developed a generative ZSL framework with Generative Adversarial Networks (GANs)
  arXiv  Detail & Related papers  (2021-02-15T04:39:58Z)
• Isometric Propagation Network for Generalized Zero-shot Learning [72.02404519815663]
  A popular strategy is to learn a mapping between the semantic space of class attributes and the visual space of images based on the seen classes and their data. We propose Isometric propagation Network (IPN), which learns to strengthen the relation between classes within each space and align the class dependency in the two spaces. IPN achieves state-of-the-art performance on three popular Zero-shot learning benchmarks.
  arXiv  Detail & Related papers  (2021-02-03T12:45:38Z)
• Attribute Propagation Network for Graph Zero-shot Learning [57.68486382473194]
  We introduce the attribute propagation network (APNet), which is composed of 1) a graph propagation model generating attribute vector for each class and 2) a parameterized nearest neighbor (NN) classifier. APNet achieves either compelling performance or new state-of-the-art results in experiments with two zero-shot learning settings and five benchmark datasets.
  arXiv  Detail & Related papers  (2020-09-24T16:53:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.