Information Bottleneck Constrained Latent Bidirectional Embedding for Zero-Shot Learning

• URL: http://arxiv.org/abs/2009.07451v3
• Date: Thu, 4 Mar 2021 07:50:10 GMT
• Title: Information Bottleneck Constrained Latent Bidirectional Embedding for Zero-Shot Learning
• Authors: Yang Liu, Lei Zhou, Xiao Bai, Lin Gu, Tatsuya Harada, Jun Zhou
• Abstract summary: We propose a novel embedding based generative model with a tight visual-semantic coupling constraint. We learn a unified latent space that calibrates the embedded parametric distributions of both visual and semantic spaces. Our method can be easily extended to transductive ZSL setting by generating labels for unseen images.
• Score: 59.58381904522967
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Zero-shot learning (ZSL) aims to recognize novel classes by transferring semantic knowledge from seen classes to unseen classes. Though many ZSL methods rely on a direct mapping between the visual and the semantic space, the calibration deviation and hubness problem limit the generalization capability to unseen classes. Recently emerged generative ZSL methods generate unseen image features to transform ZSL into a supervised classification problem. However, most generative models still suffer from the seen-unseen bias problem as only seen data is used for training. To address these issues, we propose a novel bidirectional embedding based generative model with a tight visual-semantic coupling constraint. We learn a unified latent space that calibrates the embedded parametric distributions of both visual and semantic spaces. Since the embedding from high-dimensional visual features comprise much non-semantic information, the alignment of visual and semantic in latent space would inevitably been deviated. Therefore, we introduce information bottleneck (IB) constraint to ZSL for the first time to preserve essential attribute information during the mapping. Specifically, we utilize the uncertainty estimation and the wake-sleep procedure to alleviate the feature noises and improve model abstraction capability. In addition, our method can be easily extended to transductive ZSL setting by generating labels for unseen images. We then introduce a robust loss to solve this label noise problem. Extensive experimental results show that our method outperforms the state-of-the-art methods in different ZSL settings on most benchmark datasets. The code will be available at https://github.com/osierboy/IBZSL.

Related papers

• Zero-Shot Learning by Harnessing Adversarial Samples [52.09717785644816]
  We propose a novel Zero-Shot Learning (ZSL) approach by Harnessing Adversarial Samples (HAS) HAS advances ZSL through adversarial training which takes into account three crucial aspects. We demonstrate the effectiveness of our adversarial samples approach in both ZSL and Generalized Zero-Shot Learning (GZSL) scenarios.
  arXiv  Detail & Related papers  (2023-08-01T06:19:13Z)
• Bi-directional Distribution Alignment for Transductive Zero-Shot Learning [48.80413182126543]
  We propose a novel zero-shot learning model (TZSL) called Bi-VAEGAN. It largely improves the shift by a strengthened distribution alignment between the visual and auxiliary spaces. In benchmark evaluation, Bi-VAEGAN achieves the new state of the arts under both the standard and generalized TZSL settings.
  arXiv  Detail & Related papers  (2023-03-15T15:32:59Z)
• A Simple Approach for Zero-Shot Learning based on Triplet Distribution Embeddings [6.193231258199234]
  ZSL aims to recognize unseen classes without labeled training data by exploiting semantic information. Existing ZSL methods mainly use vectors to represent the embeddings to the semantic space. We address this issue by leveraging the use of distribution embeddings.
  arXiv  Detail & Related papers  (2021-03-29T20:26:20Z)
• Zero-Shot Learning Based on Knowledge Sharing [0.0]
  Zero-Shot Learning (ZSL) is an emerging research that aims to solve the classification problems with very few training data. This paper introduces knowledge sharing (KS) to enrich the representation of semantic features. Based on KS, we apply a generative adversarial network to generate pseudo visual features from semantic features that are very close to the real visual features.
  arXiv  Detail & Related papers  (2021-02-26T06:43:29Z)
• Isometric Propagation Network for Generalized Zero-shot Learning [72.02404519815663]
  A popular strategy is to learn a mapping between the semantic space of class attributes and the visual space of images based on the seen classes and their data. We propose Isometric propagation Network (IPN), which learns to strengthen the relation between classes within each space and align the class dependency in the two spaces. IPN achieves state-of-the-art performance on three popular Zero-shot learning benchmarks.
  arXiv  Detail & Related papers  (2021-02-03T12:45:38Z)
• Generalized Zero-Shot Learning via VAE-Conditioned Generative Flow [83.27681781274406]
  Generalized zero-shot learning aims to recognize both seen and unseen classes by transferring knowledge from semantic descriptions to visual representations. Recent generative methods formulate GZSL as a missing data problem, which mainly adopts GANs or VAEs to generate visual features for unseen classes. We propose a conditional version of generative flows for GZSL, i.e., VAE-Conditioned Generative Flow (VAE-cFlow)
  arXiv  Detail & Related papers  (2020-09-01T09:12:31Z)
• Leveraging Seen and Unseen Semantic Relationships for Generative Zero-Shot Learning [14.277015352910674]
  We propose a generative model that explicitly performs knowledge transfer by incorporating a novel Semantic Regularized Loss (SR-Loss) Experiments on seven benchmark datasets demonstrate the superiority of the LsrGAN compared to previous state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-07-19T01:25:53Z)
• Generative Model-driven Structure Aligning Discriminative Embeddings for Transductive Zero-shot Learning [21.181715602603436]
  We propose a neural network-based model for learning a projection function which aligns the visual and semantic data in the latent space. We show superior performance on standard benchmark datasets AWA1, AWA2, CUB, SUN, FLO, and. We also show the efficacy of our model in the case of extremely less labelled data regime.
  arXiv  Detail & Related papers  (2020-05-09T18:48:20Z)
• Density-Aware Graph for Deep Semi-Supervised Visual Recognition [102.9484812869054]
  Semi-supervised learning (SSL) has been extensively studied to improve the generalization ability of deep neural networks for visual recognition. This paper proposes to solve the SSL problem by building a novel density-aware graph, based on which the neighborhood information can be easily leveraged.
  arXiv  Detail & Related papers  (2020-03-30T02:52:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.