Less but Better: Enabling Generalized Zero-shot Learning Towards Unseen Domains by Intrinsic Learning from Redundant LLM Semantics

• URL: http://arxiv.org/abs/2403.14362v4
• Date: Mon, 19 Aug 2024 12:28:55 GMT
• Title: Less but Better: Enabling Generalized Zero-shot Learning Towards Unseen Domains by Intrinsic Learning from Redundant LLM Semantics
• Authors: Jiaqi Yue, Jiancheng Zhao, Chunhui Zhao,
• Abstract summary: Generalized zero-shot learning (GZSL) focuses on recognizing seen and unseen classes against domain shift problem (DSP) We pioneer cross-domain GZSL which addresses GZSL towards unseen domains. Considering the information asymmetry problem caused by redundant class semantics annotated with large language models (LLMs), we present Meta Domain Alignment Semantic Refinement (MDASR)
• Score: 21.306437024924875
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generalized zero-shot learning (GZSL) focuses on recognizing seen and unseen classes against domain shift problem (DSP) where data of unseen classes may be misclassified as seen classes. However, existing GZSL is still limited to seen domains. In the current work, we pioneer cross-domain GZSL (CDGZSL) which addresses GZSL towards unseen domains. Different from existing GZSL methods which alleviate DSP by generating features of unseen classes with semantics, CDGZSL needs to construct a common feature space across domains and acquire the corresponding intrinsic semantics shared among domains to transfer from seen to unseen domains. Considering the information asymmetry problem caused by redundant class semantics annotated with large language models (LLMs), we present Meta Domain Alignment Semantic Refinement (MDASR). Technically, MDASR consists of two parts: Inter-class Similarity Alignment (ISA), which eliminates the non-intrinsic semantics not shared across all domains under the guidance of inter-class feature relationships, and Unseen-class Meta Generation (UMG), which preserves intrinsic semantics to maintain connectivity between seen and unseen classes by simulating feature generation. MDASR effectively aligns the redundant semantic space with the common feature space, mitigating the information asymmetry in CDGZSL. The effectiveness of MDASR is demonstrated on the Office-Home and Mini-DomainNet, and we have shared the LLM-based semantics for these datasets as the benchmark.

Related papers

• SemiDAViL: Semi-supervised Domain Adaptation with Vision-Language Guidance for Semantic Segmentation [9.311853182451289]
  We propose a language-guided Semi-supervised Domain Adaptation (SSDA) setting for semantic segmentation. We harness the semantic generalization capabilities inherent in vision-language models (VLMs) to establish a synergistic framework. Our approach demonstrates substantial performance improvements over contemporary state-of-the-art (SoTA) methodologies.
  arXiv  Detail & Related papers  (2025-04-08T19:14:34Z)
• LLM-RecG: A Semantic Bias-Aware Framework for Zero-Shot Sequential Recommendation [5.512301280728178]
  Zero-shot cross-domain sequential recommendation (ZCDSR) enables predictions in unseen domains without additional training or fine-tuning.<n>Recent advancements in large language models (LLMs) have significantly enhanced ZCDSR by facilitating cross-domain knowledge transfer.<n>We propose a novel semantic bias-aware framework that improves cross-domain alignment at both the item and sequential levels.
  arXiv  Detail & Related papers  (2025-01-31T15:43:21Z)
• Exploring User Retrieval Integration towards Large Language Models for Cross-Domain Sequential Recommendation [66.72195610471624]
  Cross-Domain Sequential Recommendation aims to mine and transfer users' sequential preferences across different domains. We propose a novel framework named URLLM, which aims to improve the CDSR performance by exploring the User Retrieval approach.
  arXiv  Detail & Related papers  (2024-06-05T09:19:54Z)
• Towards domain-invariant Self-Supervised Learning with Batch Styles Standardization [1.6060199783864477]
  Batch Styles Standardization (BSS) is a simple yet powerful method to standardize the style of images in a batch. We show that BSS significantly improves downstream task performances on unseen domains, often outperforming or rivaling UDG methods.
  arXiv  Detail & Related papers  (2023-03-10T17:09:04Z)
• Transferrable Contrastive Learning for Visual Domain Adaptation [108.98041306507372]
  Transferrable Contrastive Learning (TCL) is a self-supervised learning paradigm tailored for domain adaptation. TCL penalizes cross-domain intra-class domain discrepancy between source and target through a clean and novel contrastive loss. The free lunch is, thanks to the incorporation of contrastive learning, TCL relies on a moving-averaged key encoder that naturally achieves a temporally ensembled version of pseudo labels for target data.
  arXiv  Detail & Related papers  (2021-12-14T16:23:01Z)
• HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learning [74.76431541169342]
  Zero-shot learning (ZSL) tackles the unseen class recognition problem, transferring semantic knowledge from seen classes to unseen ones. We propose a novel hierarchical semantic-visual adaptation (HSVA) framework to align semantic and visual domains. Experiments on four benchmark datasets demonstrate HSVA achieves superior performance on both conventional and generalized ZSL.
  arXiv  Detail & Related papers  (2021-09-30T14:27:50Z)
• Structured Latent Embeddings for Recognizing Unseen Classes in Unseen Domains [108.11746235308046]
  We propose a novel approach that learns domain-agnostic structured latent embeddings by projecting images from different domains. Our experiments on the challenging DomainNet and DomainNet-LS benchmarks show the superiority of our approach over existing methods.
  arXiv  Detail & Related papers  (2021-07-12T17:57:46Z)
• Learning Robust Visual-semantic Mapping for Zero-shot Learning [8.299945169799795]
  We focus on fully empowering the semantic feature space, which is one of the key building blocks of Zero-shot learning (ZSL) In ZSL, the common practice is to train a mapping function between the visual and semantic feature spaces with labeled seen class examples. Under such a paradigm, the ZSL models may easily suffer from the domain shift problem when constructing and reusing the mapping function.
  arXiv  Detail & Related papers  (2021-04-12T17:39:38Z)
• OntoZSL: Ontology-enhanced Zero-shot Learning [19.87808305218359]
  Key to implementing Zero-shot Learning (ZSL) is to leverage the prior knowledge of classes which builds the semantic relationship between classes. In this paper, we explore richer and more competitive prior knowledge to model the inter-class relationship for ZSL. To address the data imbalance between seen classes and unseen classes, we developed a generative ZSL framework with Generative Adversarial Networks (GANs)
  arXiv  Detail & Related papers  (2021-02-15T04:39:58Z)
• Cross-Domain Grouping and Alignment for Domain Adaptive Semantic Segmentation [74.3349233035632]
  Existing techniques to adapt semantic segmentation networks across the source and target domains within deep convolutional neural networks (CNNs) do not consider an inter-class variation within the target domain itself or estimated category. We introduce a learnable clustering module, and a novel domain adaptation framework called cross-domain grouping and alignment. Our method consistently boosts the adaptation performance in semantic segmentation, outperforming the state-of-the-arts on various domain adaptation settings.
  arXiv  Detail & Related papers  (2020-12-15T11:36:21Z)
• Towards Recognizing Unseen Categories in Unseen Domains [74.29101415077523]
  CuMix is a holistic algorithm to tackle Zero-Shot Learning (ZSL), Domain Adaptation and Domain Generalization (DG) and ZSL+DG. The key idea of CuMix is to simulate the test-time domain and semantic shift using images and features from unseen domains and categories available during training. Results on standard SL and DG datasets and on ZSL+DG using the DomainNet benchmark demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-07-23T21:09:28Z)
• Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation [95.10255219396109]
  We propose a fine-grained adversarial learning strategy for class-level feature alignment. We adopt a fine-grained domain discriminator that not only plays as a domain distinguisher, but also differentiates domains at class level. An analysis with Class Center Distance (CCD) validates that our fine-grained adversarial strategy achieves better class-level alignment.
  arXiv  Detail & Related papers  (2020-07-17T20:50:59Z)
• Learning Cross-domain Semantic-Visual Relationships for Transductive Zero-Shot Learning [29.498249893085287]
  This work proposes the Transferrable Semantic-Visual Relation (TSVR) approach towards transductive Zero-Shot Learning (ZSL) TSVR redefines image recognition as predicting the similarity/dissimilarity labels for semantic-visual fusions consisting of class attributes and visual features. For the problem, the number of similar semantic-visual pairs is significantly smaller than that of dissimilar ones.
  arXiv  Detail & Related papers  (2020-03-31T11:26:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.