Incentivizing Generative Zero-Shot Learning via Outcome-Reward Reinforcement Learning with Visual Cues

• URL: http://arxiv.org/abs/2603.21138v1
• Date: Sun, 22 Mar 2026 09:28:04 GMT
• Title: Incentivizing Generative Zero-Shot Learning via Outcome-Reward Reinforcement Learning with Visual Cues
• Abstract summary: We propose RLVC, an outcome-reward reinforcement learning framework with visual cues for generative ZSL.<n> RLVC updates the generative model using an outcome-based reward, encouraging the synthesis of task-relevant features.<n>Experiments and analyses on three prevalent ZSL benchmarks demonstrate that RLVC achieves state-of-the-art results with a 4.7% gain.
• Score: 26.99746943267178
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in zero-shot learning (ZSL) have demonstrated the potential of generative models. Typically, generative ZSL synthesizes visual features conditioned on semantic prototypes to model the data distribution of unseen classes, followed by training a classifier on the synthesized data. However, the synthesized features often remain task-agnostic, leading to degraded performance. Moreover, inferring a faithful distribution from semantic prototypes alone is insufficient for classes that are semantically similar but visually distinct. To address these and advance ZSL, we propose RLVC, an outcome-reward reinforcement learning RL framework with visual cues for generative ZSL. At its core, RL empowers the generative model to self-evolve, implicitly enhancing its generation capability. In particular, RLVC updates the generative model using an outcome-based reward, encouraging the synthesis of task-relevant features. Furthermore, we introduce class-wise visual cues that (i) align synthesized features with visual prototypes and (ii) stabilize the RL training updates. For the training process, we present a novel cold-start strategy. Comprehensive experiments and analyses on three prevalent ZSL benchmarks demonstrate that RLVC achieves state-of-the-art results with a 4.7% gain.

Related papers

• SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning [88.9014727048442]
  SSL4RL is a novel framework that leverages self-supervised learning tasks as a source of verifiable rewards for RL-based fine-tuning.<n>Our approach reformulates SSL objectives-such as predicting image rotation or reconstructing masked patches-into dense, automatic reward signals.<n>Experiments show that SSL4RL substantially improves performance on both vision-centric and vision-language reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-10-18T09:22:40Z)
• TTS-VAR: A Test-Time Scaling Framework for Visual Auto-Regressive Generation [48.61855865678161]
  We present the first general test-time scaling framework for visual auto-regressive ( VAR) models.<n>We propose clustering-based diversity search and resampling-based potential selection.<n>Experiments on the powerful VAR model Infinity show a notable 8.7% GenEval score improvement.
  arXiv  Detail & Related papers  (2025-07-24T16:04:55Z)
• GenZSL: Generative Zero-Shot Learning Via Inductive Variational Autoencoder [56.573203512455706]
  We propose an inductive variational autoencoder for generative zero-shot learning, dubbed GenZSL.<n>Our GenZSL operates by inducting new class samples from similar seen classes using weak class semantic vectors.<n>Experiments conducted on three popular benchmark datasets showcase the superiority and potential of our GenZSL.
  arXiv  Detail & Related papers  (2025-05-17T07:24:13Z)
• Meta-Learned Attribute Self-Interaction Network for Continual and Generalized Zero-Shot Learning [46.6282595346048]
  Zero-shot learning (ZSL) is a promising approach to generalizing a model to unseen categories during training. We propose a Meta-learned Attribute self-Interaction Network (MAIN) for continual ZSL. By pairing attribute self-interaction trained using meta-learning with inverse regularization of the attribute encoder, we are able to outperform state-of-the-art results without leveraging the unseen class attributes.
  arXiv  Detail & Related papers  (2023-12-02T16:23:01Z)
• Continual Zero-Shot Learning through Semantically Guided Generative Random Walks [56.65465792750822]
  We address the challenge of continual zero-shot learning where unseen information is not provided during training, by leveraging generative modeling. We propose our learning algorithm that employs a novel semantically guided Generative Random Walk (GRW) loss. Our algorithm achieves state-of-the-art performance on AWA1, AWA2, CUB, and SUN datasets, surpassing existing CZSL methods by 3-7%.
  arXiv  Detail & Related papers  (2023-08-23T18:10:12Z)
• Synthetic Sample Selection for Generalized Zero-Shot Learning [4.264192013842096]
  Generalized Zero-Shot Learning (GZSL) has emerged as a pivotal research domain in computer vision. This paper proposes a novel approach for synthetic feature selection using reinforcement learning.
  arXiv  Detail & Related papers  (2023-04-06T03:22:43Z)
• Efficient Gaussian Process Model on Class-Imbalanced Datasets for Generalized Zero-Shot Learning [37.00463358780726]
  We propose a Neural Network model that learns a latent feature embedding and a Gaussian Process (GP) regression model that predicts latent feature prototypes of unseen classes. Our model is trained efficiently with a simple training strategy that mitigates the impact of class-imbalanced training data.
  arXiv  Detail & Related papers  (2022-10-11T04:57:20Z)
• Zero-Shot Logit Adjustment [89.68803484284408]
  Generalized Zero-Shot Learning (GZSL) is a semantic-descriptor-based learning technique. In this paper, we propose a new generation-based technique to enhance the generator's effect while neglecting the improvement of the classifier. Our experiments demonstrate that the proposed technique achieves state-of-the-art when combined with the basic generator, and it can improve various generative zero-shot learning frameworks.
  arXiv  Detail & Related papers  (2022-04-25T17:54:55Z)
• Meta-Learned Attribute Self-Gating for Continual Generalized Zero-Shot Learning [82.07273754143547]
  We propose a meta-continual zero-shot learning (MCZSL) approach to generalizing a model to categories unseen during training. By pairing self-gating of attributes and scaled class normalization with meta-learning based training, we are able to outperform state-of-the-art results.
  arXiv  Detail & Related papers  (2021-02-23T18:36:14Z)
• Generative Replay-based Continual Zero-Shot Learning [7.909034037183046]
  We develop a generative replay-based continual ZSL (GRCZSL) The proposed method endows traditional ZSL to learn from streaming data and acquire new knowledge without forgetting the previous tasks' experience. The proposed GRZSL method is developed for a single-head setting of continual learning, simulating a real-world problem setting.
  arXiv  Detail & Related papers  (2021-01-22T00:03:34Z)
• CRL: Class Representative Learning for Image Classification [5.11566193457943]
  We propose a novel model, called Class Representative Learning Model (CRL), that can be especially effective in image classification influenced by ZSL. In the CRL model, first, the learning step is to build class representatives to represent classes in datasets by aggregating prominent features extracted from a Convolutional Neural Network (CNN) The proposed CRL model demonstrated superior performance compared to the current state-of-the-art research in ZSL and mobile deep learning.
  arXiv  Detail & Related papers  (2020-02-16T17:02:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.