Mutually Causal Semantic Distillation Network for Zero-Shot Learning

• URL: http://arxiv.org/abs/2603.17412v1
• Date: Wed, 18 Mar 2026 06:44:54 GMT
• Title: Mutually Causal Semantic Distillation Network for Zero-Shot Learning
• Abstract summary: Zero-shot learning (ZSL) aims to recognize the unseen classes in the open-world guided by the side-information (e.g., attributes)<n>Its key task is how to infer the latent semantic knowledge between visual and attribute features on seen classes.<n>We propose a mutually-based attribute-based semantic network (termed++) to distill the intrinsic and sufficient semantic representations for ZSL.
• Score: 32.25476851030761
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Zero-shot learning (ZSL) aims to recognize the unseen classes in the open-world guided by the side-information (e.g., attributes). Its key task is how to infer the latent semantic knowledge between visual and attribute features on seen classes, and thus conducting a desirable semantic knowledge transfer from seen classes to unseen ones. Prior works simply utilize unidirectional attention within a weakly-supervised manner to learn the spurious and limited latent semantic representations, which fail to effectively discover the intrinsic semantic knowledge (e.g., attribute semantic) between visual and attribute features. To solve the above challenges, we propose a mutually causal semantic distillation network (termed MSDN++) to distill the intrinsic and sufficient semantic representations for ZSL. MSDN++ consists of an attribute$\rightarrow$visual causal attention sub-net that learns attribute-based visual features, and a visual$\rightarrow$attribute causal attention sub-net that learns visual-based attribute features. The causal attentions encourages the two sub-nets to learn causal vision-attribute associations for representing reliable features with causal visual/attribute learning. With the guidance of semantic distillation loss, the two mutual attention sub-nets learn collaboratively and teach each other throughout the training process. Extensive experiments on three widely-used benchmark datasets (e.g., CUB, SUN, AWA2, and FLO) show that our MSDN++ yields significant improvements over the strong baselines, leading to new state-of-the-art performances.

Related papers

• Attend and Enrich: Enhanced Visual Prompt for Zero-Shot Learning [114.59476118365266]
  We propose AENet, which endows semantic information into the visual prompt to distill semantic-enhanced prompt for visual representation enrichment.<n> AENet comprises two key steps: 1) exploring the concept-harmonized tokens for the visual and attribute modalities, grounded on the modal-sharing token that represents consistent visual-semantic concepts; and 2) yielding semantic-enhanced prompt via the visual residual refinement unit with attribute consistency supervision.
  arXiv  Detail & Related papers  (2024-06-05T07:59:48Z)
• Dual Relation Mining Network for Zero-Shot Learning [48.89161627050706]
  We propose a Dual Relation Mining Network (DRMN) to enable effective visual-semantic interactions and learn semantic relationship among attributes for knowledge transfer. Specifically, we introduce a Dual Attention Block (DAB) for visual-semantic relationship mining, which enriches visual information by multi-level feature fusion. For semantic relationship modeling, we utilize a Semantic Interaction Transformer (SIT) to enhance the generalization of attribute representations among images.
  arXiv  Detail & Related papers  (2024-05-06T16:31:19Z)
• CREST: Cross-modal Resonance through Evidential Deep Learning for Enhanced Zero-Shot Learning [48.46511584490582]
  Zero-shot learning (ZSL) enables the recognition of novel classes by leveraging semantic knowledge transfer from known to unknown categories. Real-world challenges such as distribution imbalances and attribute co-occurrence hinder the discernment of local variances in images. We propose a bidirectional cross-modal ZSL approach CREST to overcome these challenges.
  arXiv  Detail & Related papers  (2024-04-15T10:19:39Z)
• Progressive Semantic-Guided Vision Transformer for Zero-Shot Learning [56.65891462413187]
  We propose a progressive semantic-guided vision transformer for zero-shot learning (dubbed ZSLViT) ZSLViT first introduces semantic-embedded token learning to improve the visual-semantic correspondences via semantic enhancement. Then, we fuse low semantic-visual correspondence visual tokens to discard the semantic-unrelated visual information for visual enhancement.
  arXiv  Detail & Related papers  (2024-04-11T12:59:38Z)
• Attribute-Aware Representation Rectification for Generalized Zero-Shot Learning [19.65026043141699]
  Generalized Zero-shot Learning (GZSL) has yielded remarkable performance by designing a series of unbiased visual-semantics mappings. We propose a simple yet effective Attribute-Aware Representation Rectification framework for GZSL, dubbed $mathbf(AR)2$.
  arXiv  Detail & Related papers  (2023-11-23T11:30:32Z)
• Dual Feature Augmentation Network for Generalized Zero-shot Learning [14.410978100610489]
  Zero-shot learning (ZSL) aims to infer novel classes without training samples by transferring knowledge from seen classes. Existing embedding-based approaches for ZSL typically employ attention mechanisms to locate attributes on an image. We propose a novel Dual Feature Augmentation Network (DFAN), which comprises two feature augmentation modules.
  arXiv  Detail & Related papers  (2023-09-25T02:37:52Z)
• Exploiting Semantic Attributes for Transductive Zero-Shot Learning [97.61371730534258]
  Zero-shot learning aims to recognize unseen classes by generalizing the relation between visual features and semantic attributes learned from the seen classes. We present a novel transductive ZSL method that produces semantic attributes of the unseen data and imposes them on the generative process. Experiments on five standard benchmarks show that our method yields state-of-the-art results for zero-shot learning.
  arXiv  Detail & Related papers  (2023-03-17T09:09:48Z)
• MSDN: Mutually Semantic Distillation Network for Zero-Shot Learning [28.330268557106912]
  Key challenge of zero-shot learning (ZSL) is how to infer the latent semantic knowledge between visual and attribute features on seen classes. We propose a Mutually Semantic Distillation Network (MSDN), which progressively distills the intrinsic semantic representations between visual and attribute features.
  arXiv  Detail & Related papers  (2022-03-07T05:27:08Z)
• TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learning [119.43299939907685]
  Zero-shot learning (ZSL) tackles the novel class recognition problem by transferring semantic knowledge from seen classes to unseen ones. Existing attention-based models have struggled to learn inferior region features in a single image by solely using unidirectional attention. We propose a cross attribute-guided Transformer network, termed TransZero++, to refine visual features and learn accurate attribute localization for semantic-augmented visual embedding representations.
  arXiv  Detail & Related papers  (2021-12-16T05:49:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.