Improved Feature Generating Framework for Transductive Zero-shot Learning

• URL: http://arxiv.org/abs/2412.18282v1
• Date: Tue, 24 Dec 2024 08:42:16 GMT
• Title: Improved Feature Generating Framework for Transductive Zero-shot Learning
• Authors: Zihan Ye, Xinyuan Ru, Shiming Chen, Yaochu Jin, Kaizhu Huang, Xiaobo Jin,
• Abstract summary: Feature Generative Adversarial Networks have emerged as powerful generative models in producing high-quality representations of unseen classes. This paper delves into the pivotal influence of unseen class priors within the framework of Zero-shot Learning (TZSL) We introduce our Improved Feature Generation Framework, termed I-VAEGAN, which incorporates two novel components: Pseudo-conditional Feature Adversarial (PFA) learning and Variational Embedding Regression (VER)
• Score: 31.656888766677664
• License:
• Abstract: Feature Generative Adversarial Networks have emerged as powerful generative models in producing high-quality representations of unseen classes within the scope of Zero-shot Learning (ZSL). This paper delves into the pivotal influence of unseen class priors within the framework of transductive ZSL (TZSL) and illuminates the finding that even a marginal prior bias can result in substantial accuracy declines. Our extensive analysis uncovers that this inefficacy fundamentally stems from the utilization of an unconditional unseen discriminator - a core component in existing TZSL. We further establish that the detrimental effects of this component are inevitable unless the generator perfectly fits class-specific distributions. Building on these insights, we introduce our Improved Feature Generation Framework, termed I-VAEGAN, which incorporates two novel components: Pseudo-conditional Feature Adversarial (PFA) learning and Variational Embedding Regression (VER). PFA circumvents the need for prior estimation by explicitly injecting the predicted semantics as pseudo conditions for unseen classes premised by precise semantic regression. Meanwhile, VER utilizes reconstructive pre-training to learn class statistics, obtaining better semantic regression. Our I-VAEGAN achieves state-of-the-art TZSL accuracy across various benchmarks and priors. Our code would be released upon acceptance.

Related papers

• Understanding the Role of Equivariance in Self-supervised Learning [51.56331245499712]
  equivariant self-supervised learning (E-SSL) learns features to be augmentation-aware. We identify a critical explaining-away effect in E-SSL that creates a synergy between the equivariant and classification tasks. We reveal several principles for practical designs of E-SSL.
  arXiv  Detail & Related papers  (2024-11-10T16:09:47Z)
• Understanding and Mitigating Miscalibration in Prompt Tuning for Vision-Language Models [22.501089777956654]
  Confidence calibration is critical for the safe deployment of machine learning models in the real world. Existing prompt tuning methods usually lead to a trade-off of calibration between base and new classes. We introduce Dynamic Outlier Regularization to ensure the confidence calibration on both base and new classes after fine-tuning.
  arXiv  Detail & Related papers  (2024-10-03T17:06:21Z)
• Federated Class-Incremental Learning with Hierarchical Generative Prototypes [10.532838477096055]
  Federated Learning (FL) aims at unburdening the training of deep models by distributing computation across multiple devices (clients) Our proposal constrains both biases in the last layer by efficiently finetuning a pre-trained backbone using learnable prompts. Our method significantly improves the current State Of The Art, providing an average increase of +7.8% in accuracy.
  arXiv  Detail & Related papers  (2024-06-04T16:12:27Z)
• Integrating Prior Knowledge in Contrastive Learning with Kernel [4.050766659420731]
  We use kernel theory to propose a novel loss, called decoupled uniformity, that i) allows the integration of prior knowledge and ii) removes the negative-positive coupling in the original InfoNCE loss. In an unsupervised setting, we empirically demonstrate that CL benefits from generative models to improve its representation both on natural and medical images.
  arXiv  Detail & Related papers  (2022-06-03T15:43:08Z)
• Zero-Shot Logit Adjustment [89.68803484284408]
  Generalized Zero-Shot Learning (GZSL) is a semantic-descriptor-based learning technique. In this paper, we propose a new generation-based technique to enhance the generator's effect while neglecting the improvement of the classifier. Our experiments demonstrate that the proposed technique achieves state-of-the-art when combined with the basic generator, and it can improve various generative zero-shot learning frameworks.
  arXiv  Detail & Related papers  (2022-04-25T17:54:55Z)
• Towards the Semantic Weak Generalization Problem in Generative Zero-Shot Learning: Ante-hoc and Post-hoc [89.68803484284408]
  We present a simple and effective strategy lowering the previously unexplored factors that limit the performance ceiling of generative Zero-Shot Learning (ZSL) We begin by formally defining semantic generalization, then look into approaches for reducing the semantic weak generalization problem. In the ante-hoc phase, we augment the generator's semantic input, as well as relax the fitting target of the generator.
  arXiv  Detail & Related papers  (2022-04-24T13:54:42Z)
• Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalization [52.7137956951533]
  We argue that devising simpler methods for learning predictors on existing features is a promising direction for future research. We introduce Domain-Adjusted Regression (DARE), a convex objective for learning a linear predictor that is provably robust under a new model of distribution shift. Under a natural model, we prove that the DARE solution is the minimax-optimal predictor for a constrained set of test distributions.
  arXiv  Detail & Related papers  (2022-02-14T16:42:16Z)
• Structure-Aware Feature Generation for Zero-Shot Learning [108.76968151682621]
  We introduce a novel structure-aware feature generation scheme, termed as SA-GAN, to account for the topological structure in learning both the latent space and the generative networks. Our method significantly enhances the generalization capability on unseen-classes and consequently improve the classification performance.
  arXiv  Detail & Related papers  (2021-08-16T11:52:08Z)
• Generalized Zero-Shot Learning via VAE-Conditioned Generative Flow [83.27681781274406]
  Generalized zero-shot learning aims to recognize both seen and unseen classes by transferring knowledge from semantic descriptions to visual representations. Recent generative methods formulate GZSL as a missing data problem, which mainly adopts GANs or VAEs to generate visual features for unseen classes. We propose a conditional version of generative flows for GZSL, i.e., VAE-Conditioned Generative Flow (VAE-cFlow)
  arXiv  Detail & Related papers  (2020-09-01T09:12:31Z)
• Invertible Zero-Shot Recognition Flows [42.839333265321905]
  This work incorporates a new family of generative models (i.e., flow-based models) into Zero-Shot Learning (ZSL) The proposed Invertible Zero-shot Flow (IZF) learns factorized data embeddings with the forward pass of an invertible flow network, while the reverse pass generates data samples. Experiments on widely-adopted ZSL benchmarks demonstrate the significant performance gain of IZF over existing methods.
  arXiv  Detail & Related papers  (2020-07-09T15:21:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.