Generalized Zero-Shot Learning via VAE-Conditioned Generative Flow

• URL: http://arxiv.org/abs/2009.00303v1
• Date: Tue, 1 Sep 2020 09:12:31 GMT
• Title: Generalized Zero-Shot Learning via VAE-Conditioned Generative Flow
• Authors: Yu-Chao Gu, Le Zhang, Yun Liu, Shao-Ping Lu, Ming-Ming Cheng
• Abstract summary: Generalized zero-shot learning aims to recognize both seen and unseen classes by transferring knowledge from semantic descriptions to visual representations. Recent generative methods formulate GZSL as a missing data problem, which mainly adopts GANs or VAEs to generate visual features for unseen classes. We propose a conditional version of generative flows for GZSL, i.e., VAE-Conditioned Generative Flow (VAE-cFlow)
• Score: 83.27681781274406
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generalized zero-shot learning (GZSL) aims to recognize both seen and unseen classes by transferring knowledge from semantic descriptions to visual representations. Recent generative methods formulate GZSL as a missing data problem, which mainly adopts GANs or VAEs to generate visual features for unseen classes. However, GANs often suffer from instability, and VAEs can only optimize the lower bound on the log-likelihood of observed data. To overcome the above limitations, we resort to generative flows, a family of generative models with the advantage of accurate likelihood estimation. More specifically, we propose a conditional version of generative flows for GZSL, i.e., VAE-Conditioned Generative Flow (VAE-cFlow). By using VAE, the semantic descriptions are firstly encoded into tractable latent distributions, conditioned on that the generative flow optimizes the exact log-likelihood of the observed visual features. We ensure the conditional latent distribution to be both semantic meaningful and inter-class discriminative by i) adopting the VAE reconstruction objective, ii) releasing the zero-mean constraint in VAE posterior regularization, and iii) adding a classification regularization on the latent variables. Our method achieves state-of-the-art GZSL results on five well-known benchmark datasets, especially for the significant improvement in the large-scale setting. Code is released at https://github.com/guyuchao/VAE-cFlow-ZSL.

Related papers

• Exploring Data Efficiency in Zero-Shot Learning with Diffusion Models [38.36200871549062]
  Zero-Shot Learning (ZSL) aims to enable classifiers to identify unseen classes by enhancing data efficiency at the class level. This is achieved by generating image features from pre-defined semantics of unseen classes. In this paper, we demonstrate that limited seen examples generally result in deteriorated performance of generative models. This unified framework incorporates diffusion models to improve data efficiency at both the class and instance levels.
  arXiv  Detail & Related papers  (2024-06-05T04:37:06Z)
• Visual-Augmented Dynamic Semantic Prototype for Generative Zero-Shot Learning [56.16593809016167]
  We propose a novel Visual-Augmented Dynamic Semantic prototype method (termed VADS) to boost the generator to learn accurate semantic-visual mapping. VADS consists of two modules: (1) Visual-aware Domain Knowledge Learning module (VDKL) learns the local bias and global prior of the visual features, which replace pure Gaussian noise to provide richer prior noise information; (2) Vision-Oriented Semantic Updation module (VOSU) updates the semantic prototype according to the visual representations of the samples.
  arXiv  Detail & Related papers  (2024-04-23T07:39:09Z)
• GSMFlow: Generation Shifts Mitigating Flow for Generalized Zero-Shot Learning [55.79997930181418]
  Generalized Zero-Shot Learning aims to recognize images from both the seen and unseen classes by transferring semantic knowledge from seen to unseen classes. It is a promising solution to take the advantage of generative models to hallucinate realistic unseen samples based on the knowledge learned from the seen classes. We propose a novel flow-based generative framework that consists of multiple conditional affine coupling layers for learning unseen data generation.
  arXiv  Detail & Related papers  (2022-07-05T04:04:37Z)
• Zero-Shot Logit Adjustment [89.68803484284408]
  Generalized Zero-Shot Learning (GZSL) is a semantic-descriptor-based learning technique. In this paper, we propose a new generation-based technique to enhance the generator's effect while neglecting the improvement of the classifier. Our experiments demonstrate that the proposed technique achieves state-of-the-art when combined with the basic generator, and it can improve various generative zero-shot learning frameworks.
  arXiv  Detail & Related papers  (2022-04-25T17:54:55Z)
• FREE: Feature Refinement for Generalized Zero-Shot Learning [86.41074134041394]
  Generalized zero-shot learning (GZSL) has achieved significant progress, with many efforts dedicated to overcoming the problems of visual-semantic domain gap and seen-unseen bias. Most existing methods directly use feature extraction models trained on ImageNet alone, ignoring the cross-dataset bias between ImageNet and GZSL benchmarks. We propose a simple yet effective GZSL method, termed feature refinement for generalized zero-shot learning (FREE) to tackle the above problem.
  arXiv  Detail & Related papers  (2021-07-29T08:11:01Z)
• Information Bottleneck Constrained Latent Bidirectional Embedding for Zero-Shot Learning [59.58381904522967]
  We propose a novel embedding based generative model with a tight visual-semantic coupling constraint. We learn a unified latent space that calibrates the embedded parametric distributions of both visual and semantic spaces. Our method can be easily extended to transductive ZSL setting by generating labels for unseen images.
  arXiv  Detail & Related papers  (2020-09-16T03:54:12Z)
• Leveraging Seen and Unseen Semantic Relationships for Generative Zero-Shot Learning [14.277015352910674]
  We propose a generative model that explicitly performs knowledge transfer by incorporating a novel Semantic Regularized Loss (SR-Loss) Experiments on seven benchmark datasets demonstrate the superiority of the LsrGAN compared to previous state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-07-19T01:25:53Z)
• Invertible Zero-Shot Recognition Flows [42.839333265321905]
  This work incorporates a new family of generative models (i.e., flow-based models) into Zero-Shot Learning (ZSL) The proposed Invertible Zero-shot Flow (IZF) learns factorized data embeddings with the forward pass of an invertible flow network, while the reverse pass generates data samples. Experiments on widely-adopted ZSL benchmarks demonstrate the significant performance gain of IZF over existing methods.
  arXiv  Detail & Related papers  (2020-07-09T15:21:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.