Learning Adversarial Semantic Embeddings for Zero-Shot Recognition in Open Worlds

• URL: http://arxiv.org/abs/2307.03416v1
• Date: Fri, 7 Jul 2023 06:54:21 GMT
• Title: Learning Adversarial Semantic Embeddings for Zero-Shot Recognition in Open Worlds
• Authors: Tianqi Li, Guansong Pang, Xiao Bai, Jin Zheng, Lei Zhou, Xin Ning
• Abstract summary: Zero-Shot Learning (ZSL) focuses on classifying samples of unseen classes with only their side semantic information presented during training. "Zero-Shot Open-Set Recognition" (ZS-OSR) is required to accurately classify samples from the unseen classes while rejecting samples from the unknown classes during inference. We introduce a novel approach specifically designed for ZS-OSR, in which our model learns to generate adversarial semantic embeddings of the unknown classes to train an unknowns-informed ZS-OSR.
• Score: 25.132219723741024
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Zero-Shot Learning (ZSL) focuses on classifying samples of unseen classes with only their side semantic information presented during training. It cannot handle real-life, open-world scenarios where there are test samples of unknown classes for which neither samples (e.g., images) nor their side semantic information is known during training. Open-Set Recognition (OSR) is dedicated to addressing the unknown class issue, but existing OSR methods are not designed to model the semantic information of the unseen classes. To tackle this combined ZSL and OSR problem, we consider the case of "Zero-Shot Open-Set Recognition" (ZS-OSR), where a model is trained under the ZSL setting but it is required to accurately classify samples from the unseen classes while being able to reject samples from the unknown classes during inference. We perform large experiments on combining existing state-of-the-art ZSL and OSR models for the ZS-OSR task on four widely used datasets adapted from the ZSL task, and reveal that ZS-OSR is a non-trivial task as the simply combined solutions perform badly in distinguishing the unseen-class and unknown-class samples. We further introduce a novel approach specifically designed for ZS-OSR, in which our model learns to generate adversarial semantic embeddings of the unknown classes to train an unknowns-informed ZS-OSR classifier. Extensive empirical results show that our method 1) substantially outperforms the combined solutions in detecting the unknown classes while retaining the classification accuracy on the unseen classes and 2) achieves similar superiority under generalized ZS-OSR settings.

Related papers

• Robust Semi-Supervised Learning for Self-learning Open-World Classes [5.714673612282175]
  In real-world applications, unlabeled data always contain classes not present in the labeled set. We propose an open-world SSL method for Self-learning Open-world Classes (SSOC), which can explicitly self-learn multiple unknown classes. SSOC outperforms the state-of-the-art baselines on multiple popular classification benchmarks.
  arXiv  Detail & Related papers  (2024-01-15T09:27:46Z)
• Semi-Supervised Learning in the Few-Shot Zero-Shot Scenario [14.916971861796384]
  Semi-Supervised Learning (SSL) is a framework that utilizes both labeled and unlabeled data to enhance model performance. We propose a general approach to augment existing SSL methods, enabling them to handle situations where certain classes are missing. Our experimental results reveal significant improvements in accuracy when compared to state-of-the-art SSL, open-set SSL, and open-world SSL methods.
  arXiv  Detail & Related papers  (2023-08-27T14:25:07Z)
• On the link between generative semi-supervised learning and generative open-set recognition [0.0]
  We investigate the relationship between semi-supervised learning (SSL) and open-set recognition (OSR) under the context of generative adversarial networks (GANs) Our results find that SSL-GANs achieve near identical results to OSR-GANs, proving the SSL-OSR link.
  arXiv  Detail & Related papers  (2023-03-21T09:42:27Z)
• The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognition [61.28722817272917]
  Open-set Recognition (OSR) aims to identify test samples whose classes are not seen during the training process. Recently, Unified Open-set Recognition (UOSR) has been proposed to reject not only unknown samples but also known but wrongly classified samples.
  arXiv  Detail & Related papers  (2023-02-08T11:34:04Z)
• Feature Decoupling in Self-supervised Representation Learning for Open Set Recognition [0.0]
  We use a two-stage training strategy for the open set recognition (OSR) problems. In the first stage, we introduce a self-supervised feature decoupling method that finds the content features of the input samples from the known classes. In the second stage, we fine-tune the content features with the class labels. Our experimental results indicate that our proposed self-supervised approach outperforms others in image and malware OSR problems.
  arXiv  Detail & Related papers  (2022-09-28T19:21:53Z)
• OpenLDN: Learning to Discover Novel Classes for Open-World Semi-Supervised Learning [110.40285771431687]
  Semi-supervised learning (SSL) is one of the dominant approaches to address the annotation bottleneck of supervised learning. Recent SSL methods can effectively leverage a large repository of unlabeled data to improve performance while relying on a small set of labeled data. This work introduces OpenLDN that utilizes a pairwise similarity loss to discover novel classes.
  arXiv  Detail & Related papers  (2022-07-05T18:51:05Z)
• Trash to Treasure: Harvesting OOD Data with Cross-Modal Matching for Open-Set Semi-Supervised Learning [101.28281124670647]
  Open-set semi-supervised learning (open-set SSL) investigates a challenging but practical scenario where out-of-distribution (OOD) samples are contained in the unlabeled data. We propose a novel training mechanism that could effectively exploit the presence of OOD data for enhanced feature learning. Our approach substantially lifts the performance on open-set SSL and outperforms the state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2021-08-12T09:14:44Z)
• Counterfactual Zero-Shot and Open-Set Visual Recognition [95.43275761833804]
  We present a novel counterfactual framework for both Zero-Shot Learning (ZSL) and Open-Set Recognition (OSR) Our idea stems from the observation that the generated samples for unseen-classes are often out of the true distribution. We demonstrate that our framework effectively mitigates the seen/unseen imbalance and hence significantly improves the overall performance.
  arXiv  Detail & Related papers  (2021-03-01T10:20:04Z)
• Generalized Continual Zero-Shot Learning [7.097782028036196]
  zero-shot learning (ZSL) aims to classify unseen classes by transferring the knowledge from seen classes to unseen classes based on the class description. We propose a more general and practical setup for ZSL, where classes arrive sequentially in the form of a task. We use knowledge distillation and storing and replay the few samples from previous tasks using a small episodic memory.
  arXiv  Detail & Related papers  (2020-11-17T08:47:54Z)
• Information Bottleneck Constrained Latent Bidirectional Embedding for Zero-Shot Learning [59.58381904522967]
  We propose a novel embedding based generative model with a tight visual-semantic coupling constraint. We learn a unified latent space that calibrates the embedded parametric distributions of both visual and semantic spaces. Our method can be easily extended to transductive ZSL setting by generating labels for unseen images.
  arXiv  Detail & Related papers  (2020-09-16T03:54:12Z)
• Open Set Recognition with Conditional Probabilistic Generative Models [51.40872765917125]
  We propose Conditional Probabilistic Generative Models (CPGM) for open set recognition. CPGM can detect unknown samples but also classify known classes by forcing different latent features to approximate conditional Gaussian distributions. Experiment results on multiple benchmark datasets reveal that the proposed method significantly outperforms the baselines.
  arXiv  Detail & Related papers  (2020-08-12T06:23:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.