Generative Model-driven Structure Aligning Discriminative Embeddings for Transductive Zero-shot Learning

• URL: http://arxiv.org/abs/2005.04492v1
• Date: Sat, 9 May 2020 18:48:20 GMT
• Title: Generative Model-driven Structure Aligning Discriminative Embeddings for Transductive Zero-shot Learning
• Authors: Omkar Gune, Mainak Pal, Preeti Mukherjee, Biplab Banerjee and Subhasis Chaudhuri
• Abstract summary: We propose a neural network-based model for learning a projection function which aligns the visual and semantic data in the latent space. We show superior performance on standard benchmark datasets AWA1, AWA2, CUB, SUN, FLO, and. We also show the efficacy of our model in the case of extremely less labelled data regime.
• Score: 21.181715602603436
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Zero-shot Learning (ZSL) is a transfer learning technique which aims at transferring knowledge from seen classes to unseen classes. This knowledge transfer is possible because of underlying semantic space which is common to seen and unseen classes. Most existing approaches learn a projection function using labelled seen class data which maps visual data to semantic data. In this work, we propose a shallow but effective neural network-based model for learning such a projection function which aligns the visual and semantic data in the latent space while simultaneously making the latent space embeddings discriminative. As the above projection function is learned using the seen class data, the so-called projection domain shift exists. We propose a transductive approach to reduce the effect of domain shift, where we utilize unlabeled visual data from unseen classes to generate corresponding semantic features for unseen class visual samples. While these semantic features are initially generated using a conditional variational auto-encoder, they are used along with the seen class data to improve the projection function. We experiment on both inductive and transductive setting of ZSL and generalized ZSL and show superior performance on standard benchmark datasets AWA1, AWA2, CUB, SUN, FLO, and APY. We also show the efficacy of our model in the case of extremely less labelled data regime on different datasets in the context of ZSL.

Related papers

• An Integral Projection-based Semantic Autoencoder for Zero-Shot Learning [0.46644955105516456]
  Zero-shot Learning (ZSL) classification categorizes or predicts classes (labels) that are not included in the training set (unseen classes) Recent works proposed different semantic autoencoder (SAE) models where the encoder embeds a visual feature space into the semantic space and the decoder reconstructs the original visual feature space. We propose an integral projection-based semantic autoencoder (IP-SAE) where an encoder projects a visual feature space vectord with the semantic space into a latent representation space.
  arXiv  Detail & Related papers  (2023-06-26T12:06:20Z)
• Exploiting Semantic Attributes for Transductive Zero-Shot Learning [97.61371730534258]
  Zero-shot learning aims to recognize unseen classes by generalizing the relation between visual features and semantic attributes learned from the seen classes. We present a novel transductive ZSL method that produces semantic attributes of the unseen data and imposes them on the generative process. Experiments on five standard benchmarks show that our method yields state-of-the-art results for zero-shot learning.
  arXiv  Detail & Related papers  (2023-03-17T09:09:48Z)
• Linking data separation, visual separation, and classifier performance using pseudo-labeling by contrastive learning [125.99533416395765]
  We argue that the performance of the final classifier depends on the data separation present in the latent space and visual separation present in the projection. We demonstrate our results by the classification of five real-world challenging image datasets of human intestinal parasites with only 1% supervised samples.
  arXiv  Detail & Related papers  (2023-02-06T10:01:38Z)
• GSMFlow: Generation Shifts Mitigating Flow for Generalized Zero-Shot Learning [55.79997930181418]
  Generalized Zero-Shot Learning aims to recognize images from both the seen and unseen classes by transferring semantic knowledge from seen to unseen classes. It is a promising solution to take the advantage of generative models to hallucinate realistic unseen samples based on the knowledge learned from the seen classes. We propose a novel flow-based generative framework that consists of multiple conditional affine coupling layers for learning unseen data generation.
  arXiv  Detail & Related papers  (2022-07-05T04:04:37Z)
• Cross-modal Representation Learning for Zero-shot Action Recognition [67.57406812235767]
  We present a cross-modal Transformer-based framework, which jointly encodes video data and text labels for zero-shot action recognition (ZSAR) Our model employs a conceptually new pipeline by which visual representations are learned in conjunction with visual-semantic associations in an end-to-end manner. Experiment results show our model considerably improves upon the state of the arts in ZSAR, reaching encouraging top-1 accuracy on UCF101, HMDB51, and ActivityNet benchmark datasets.
  arXiv  Detail & Related papers  (2022-05-03T17:39:27Z)
• Mitigating Generation Shifts for Generalized Zero-Shot Learning [52.98182124310114]
  Generalized Zero-Shot Learning (GZSL) is the task of leveraging semantic information (e.g., attributes) to recognize the seen and unseen samples, where unseen classes are not observable during training. We propose a novel Generation Shifts Mitigating Flow framework for learning unseen data synthesis efficiently and effectively. Experimental results demonstrate that GSMFlow achieves state-of-the-art recognition performance in both conventional and generalized zero-shot settings.
  arXiv  Detail & Related papers  (2021-07-07T11:43:59Z)
• Learning Robust Visual-semantic Mapping for Zero-shot Learning [8.299945169799795]
  We focus on fully empowering the semantic feature space, which is one of the key building blocks of Zero-shot learning (ZSL) In ZSL, the common practice is to train a mapping function between the visual and semantic feature spaces with labeled seen class examples. Under such a paradigm, the ZSL models may easily suffer from the domain shift problem when constructing and reusing the mapping function.
  arXiv  Detail & Related papers  (2021-04-12T17:39:38Z)
• Transductive Zero-Shot Learning by Decoupled Feature Generation [30.664199050468472]
  We focus on the transductive setting, in which unlabelled visual data from unseen classes is available. We propose to decouple tasks of generating realistic visual features and translating semantic attributes into visual cues. We present a detailed ablation study to dissect the effect of our proposed decoupling approach, while demonstrating its superiority over the related state-of-the-art.
  arXiv  Detail & Related papers  (2021-02-05T16:17:52Z)
• Information Bottleneck Constrained Latent Bidirectional Embedding for Zero-Shot Learning [59.58381904522967]
  We propose a novel embedding based generative model with a tight visual-semantic coupling constraint. We learn a unified latent space that calibrates the embedded parametric distributions of both visual and semantic spaces. Our method can be easily extended to transductive ZSL setting by generating labels for unseen images.
  arXiv  Detail & Related papers  (2020-09-16T03:54:12Z)
• A Novel Perspective to Zero-shot Learning: Towards an Alignment of Manifold Structures via Semantic Feature Expansion [17.48923061278128]
  A common practice in zero-shot learning is to train a projection between the visual and semantic feature spaces with labeled seen classes examples. Under such a paradigm, most existing methods easily suffer from the domain shift problem and weaken the performance of zero-shot recognition. We propose a novel model called AMS-SFE that considers the alignment of manifold structures by semantic feature expansion.
  arXiv  Detail & Related papers  (2020-04-30T14:08:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.