Dual Progressive Prototype Network for Generalized Zero-Shot Learning

• URL: http://arxiv.org/abs/2111.02073v1
• Date: Wed, 3 Nov 2021 08:43:29 GMT
• Title: Dual Progressive Prototype Network for Generalized Zero-Shot Learning
• Authors: Chaoqun Wang, Shaobo Min, Xuejin Chen, Xiaoyan Sun, Houqiang Li
• Abstract summary: Generalized Zero-Shot Learning (GZSL) aims to recognize new categories with auxiliary semantic information,e.g., category attributes. Our approach, named Dual Progressive Prototype Network (DPPN), constructs two types of prototypes that record visual patterns for attributes and categories. Experiments on four benchmarks prove that DPPN effectively alleviates the domain shift problem in GZSL.
• Score: 77.0715029826957
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generalized Zero-Shot Learning (GZSL) aims to recognize new categories with auxiliary semantic information,e.g., category attributes. In this paper, we handle the critical issue of domain shift problem, i.e., confusion between seen and unseen categories, by progressively improving cross-domain transferability and category discriminability of visual representations. Our approach, named Dual Progressive Prototype Network (DPPN), constructs two types of prototypes that record prototypical visual patterns for attributes and categories, respectively. With attribute prototypes, DPPN alternately searches attribute-related local regions and updates corresponding attribute prototypes to progressively explore accurate attribute-region correspondence. This enables DPPN to produce visual representations with accurate attribute localization ability, which benefits the semantic-visual alignment and representation transferability. Besides, along with progressive attribute localization, DPPN further projects category prototypes into multiple spaces to progressively repel visual representations from different categories, which boosts category discriminability. Both attribute and category prototypes are collaboratively learned in a unified framework, which makes visual representations of DPPN transferable and distinctive. Experiments on four benchmarks prove that DPPN effectively alleviates the domain shift problem in GZSL.

Related papers

• Dual Relation Mining Network for Zero-Shot Learning [48.89161627050706]
  We propose a Dual Relation Mining Network (DRMN) to enable effective visual-semantic interactions and learn semantic relationship among attributes for knowledge transfer. Specifically, we introduce a Dual Attention Block (DAB) for visual-semantic relationship mining, which enriches visual information by multi-level feature fusion. For semantic relationship modeling, we utilize a Semantic Interaction Transformer (SIT) to enhance the generalization of attribute representations among images.
  arXiv  Detail & Related papers  (2024-05-06T16:31:19Z)
• `Eyes of a Hawk and Ears of a Fox': Part Prototype Network for Generalized Zero-Shot Learning [47.1040786932317]
  Current approaches in Generalized Zero-Shot Learning (GZSL) are built upon base models which consider only a single class attribute vector representation over the entire image. We take a fundamentally different approach: a pre-trained Vision-Language detector (VINVL) sensitive to attribute information is employed to efficiently obtain region features. A learned function maps the region features to region-specific attribute attention used to construct class part prototypes.
  arXiv  Detail & Related papers  (2024-04-12T18:37:00Z)
• Dual Feature Augmentation Network for Generalized Zero-shot Learning [14.410978100610489]
  Zero-shot learning (ZSL) aims to infer novel classes without training samples by transferring knowledge from seen classes. Existing embedding-based approaches for ZSL typically employ attention mechanisms to locate attributes on an image. We propose a novel Dual Feature Augmentation Network (DFAN), which comprises two feature augmentation modules.
  arXiv  Detail & Related papers  (2023-09-25T02:37:52Z)
• Progressive Semantic-Visual Mutual Adaption for Generalized Zero-Shot Learning [74.48337375174297]
  Generalized Zero-Shot Learning (GZSL) identifies unseen categories by knowledge transferred from the seen domain. We deploy the dual semantic-visual transformer module (DSVTM) to progressively model the correspondences between prototypes and visual features. DSVTM devises an instance-motivated semantic encoder that learns instance-centric prototypes to adapt to different images, enabling the recast of the unmatched semantic-visual pair into the matched one.
  arXiv  Detail & Related papers  (2023-03-27T15:21:43Z)
• Dual Prototypical Contrastive Learning for Few-shot Semantic Segmentation [55.339405417090084]
  We propose a dual prototypical contrastive learning approach tailored to the few-shot semantic segmentation (FSS) task. The main idea is to encourage the prototypes more discriminative by increasing inter-class distance while reducing intra-class distance in prototype feature space. We demonstrate that the proposed dual contrastive learning approach outperforms state-of-the-art FSS methods on PASCAL-5i and COCO-20i datasets.
  arXiv  Detail & Related papers  (2021-11-09T08:14:50Z)
• Isometric Propagation Network for Generalized Zero-shot Learning [72.02404519815663]
  A popular strategy is to learn a mapping between the semantic space of class attributes and the visual space of images based on the seen classes and their data. We propose Isometric propagation Network (IPN), which learns to strengthen the relation between classes within each space and align the class dependency in the two spaces. IPN achieves state-of-the-art performance on three popular Zero-shot learning benchmarks.
  arXiv  Detail & Related papers  (2021-02-03T12:45:38Z)
• Cross-domain Detection via Graph-induced Prototype Alignment [114.8952035552862]
  We propose a Graph-induced Prototype Alignment (GPA) framework to seek for category-level domain alignment. In addition, in order to alleviate the negative effect of class-imbalance on domain adaptation, we design a Class-reweighted Contrastive Loss. Our approach outperforms existing methods with a remarkable margin.
  arXiv  Detail & Related papers  (2020-03-28T17:46:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.