GSMFlow: Generation Shifts Mitigating Flow for Generalized Zero-Shot Learning

• URL: http://arxiv.org/abs/2207.01798v1
• Date: Tue, 5 Jul 2022 04:04:37 GMT
• Title: GSMFlow: Generation Shifts Mitigating Flow for Generalized Zero-Shot Learning
• Authors: Zhi Chen, Yadan Luo, Ruihong Qiu, Sen Wang, Zi Huang, Jingjing Li, Zheng Zhang
• Abstract summary: Generalized Zero-Shot Learning aims to recognize images from both the seen and unseen classes by transferring semantic knowledge from seen to unseen classes. It is a promising solution to take the advantage of generative models to hallucinate realistic unseen samples based on the knowledge learned from the seen classes. We propose a novel flow-based generative framework that consists of multiple conditional affine coupling layers for learning unseen data generation.
• Score: 55.79997930181418
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Generalized Zero-Shot Learning (GZSL) aims to recognize images from both the seen and unseen classes by transferring semantic knowledge from seen to unseen classes. It is a promising solution to take the advantage of generative models to hallucinate realistic unseen samples based on the knowledge learned from the seen classes. However, due to the generation shifts, the synthesized samples by most existing methods may drift from the real distribution of the unseen data. To address this issue, we propose a novel flow-based generative framework that consists of multiple conditional affine coupling layers for learning unseen data generation. Specifically, we discover and address three potential problems that trigger the generation shifts, i.e., semantic inconsistency, variance collapse, and structure disorder. First, to enhance the reflection of the semantic information in the generated samples, we explicitly embed the semantic information into the transformation in each conditional affine coupling layer. Second, to recover the intrinsic variance of the real unseen features, we introduce a boundary sample mining strategy with entropy maximization to discover more difficult visual variants of semantic prototypes and hereby adjust the decision boundary of the classifiers. Third, a relative positioning strategy is proposed to revise the attribute embeddings, guiding them to fully preserve the inter-class geometric structure and further avoid structure disorder in the semantic space. Extensive experimental results on four GZSL benchmark datasets demonstrate that GSMFlow achieves the state-of-the-art performance on GZSL.

Related papers

• Exploring Data Efficiency in Zero-Shot Learning with Diffusion Models [38.36200871549062]
  Zero-Shot Learning (ZSL) aims to enable classifiers to identify unseen classes by enhancing data efficiency at the class level. This is achieved by generating image features from pre-defined semantics of unseen classes. In this paper, we demonstrate that limited seen examples generally result in deteriorated performance of generative models. This unified framework incorporates diffusion models to improve data efficiency at both the class and instance levels.
  arXiv  Detail & Related papers  (2024-06-05T04:37:06Z)
• Instance Adaptive Prototypical Contrastive Embedding for Generalized Zero Shot Learning [11.720039414872296]
  Generalized zero-shot learning aims to classify samples from seen and unseen labels, assuming unseen labels are not accessible during training. Recent advancements in GZSL have been expedited by incorporating contrastive-learning-based embedding in generative networks.
  arXiv  Detail & Related papers  (2023-09-13T14:26:03Z)
• Zero-Shot Logit Adjustment [89.68803484284408]
  Generalized Zero-Shot Learning (GZSL) is a semantic-descriptor-based learning technique. In this paper, we propose a new generation-based technique to enhance the generator's effect while neglecting the improvement of the classifier. Our experiments demonstrate that the proposed technique achieves state-of-the-art when combined with the basic generator, and it can improve various generative zero-shot learning frameworks.
  arXiv  Detail & Related papers  (2022-04-25T17:54:55Z)
• Prototypical Model with Novel Information-theoretic Loss Function for Generalized Zero Shot Learning [3.870962269034544]
  Generalized zero shot learning (GZSL) is still a technical challenge of deep learning. We address the quantification of the knowledge transfer and semantic relation from an information-theoretic viewpoint. We propose three information-theoretic loss functions for deterministic GZSL model.
  arXiv  Detail & Related papers  (2021-12-06T16:01:46Z)
• Structure-Aware Feature Generation for Zero-Shot Learning [108.76968151682621]
  We introduce a novel structure-aware feature generation scheme, termed as SA-GAN, to account for the topological structure in learning both the latent space and the generative networks. Our method significantly enhances the generalization capability on unseen-classes and consequently improve the classification performance.
  arXiv  Detail & Related papers  (2021-08-16T11:52:08Z)
• Mitigating Generation Shifts for Generalized Zero-Shot Learning [52.98182124310114]
  Generalized Zero-Shot Learning (GZSL) is the task of leveraging semantic information (e.g., attributes) to recognize the seen and unseen samples, where unseen classes are not observable during training. We propose a novel Generation Shifts Mitigating Flow framework for learning unseen data synthesis efficiently and effectively. Experimental results demonstrate that GSMFlow achieves state-of-the-art recognition performance in both conventional and generalized zero-shot settings.
  arXiv  Detail & Related papers  (2021-07-07T11:43:59Z)
• Generalized Zero-Shot Learning via VAE-Conditioned Generative Flow [83.27681781274406]
  Generalized zero-shot learning aims to recognize both seen and unseen classes by transferring knowledge from semantic descriptions to visual representations. Recent generative methods formulate GZSL as a missing data problem, which mainly adopts GANs or VAEs to generate visual features for unseen classes. We propose a conditional version of generative flows for GZSL, i.e., VAE-Conditioned Generative Flow (VAE-cFlow)
  arXiv  Detail & Related papers  (2020-09-01T09:12:31Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.