Open Set Recognition using Vision Transformer with an Additional Detection Head

• URL: http://arxiv.org/abs/2203.08441v1
• Date: Wed, 16 Mar 2022 07:34:58 GMT
• Title: Open Set Recognition using Vision Transformer with an Additional Detection Head
• Authors: Feiyang Cai, Zhenkai Zhang, Jie Liu, Xenofon Koutsoukos
• Abstract summary: We propose a novel approach to open set recognition (OSR) based on the vision transformer (ViT) technique. Our approach employs two separate training stages. First, a ViT model is trained to perform closed set classification. Then, an additional detection head is attached to the embedded features extracted by the ViT, trained to force the representations of known data to class-specific clusters compactly.
• Score: 6.476341388938684
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep neural networks have demonstrated prominent capacities for image classification tasks in a closed set setting, where the test data come from the same distribution as the training data. However, in a more realistic open set scenario, traditional classifiers with incomplete knowledge cannot tackle test data that are not from the training classes. Open set recognition (OSR) aims to address this problem by both identifying unknown classes and distinguishing known classes simultaneously. In this paper, we propose a novel approach to OSR that is based on the vision transformer (ViT) technique. Specifically, our approach employs two separate training stages. First, a ViT model is trained to perform closed set classification. Then, an additional detection head is attached to the embedded features extracted by the ViT, trained to force the representations of known data to class-specific clusters compactly. Test examples are identified as known or unknown based on their distance to the cluster centers. To the best of our knowledge, this is the first time to leverage ViT for the purpose of OSR, and our extensive evaluation against several OSR benchmark datasets reveals that our approach significantly outperforms other baseline methods and obtains new state-of-the-art performance.

Related papers

• Activate and Reject: Towards Safe Domain Generalization under Category Shift [71.95548187205736]
  We study a practical problem of Domain Generalization under Category Shift (DGCS) It aims to simultaneously detect unknown-class samples and classify known-class samples in the target domains. Compared to prior DG works, we face two new challenges: 1) how to learn the concept of unknown'' during training with only source known-class samples, and 2) how to adapt the source-trained model to unseen environments.
  arXiv  Detail & Related papers  (2023-10-07T07:53:12Z)
• LORD: Leveraging Open-Set Recognition with Unknown Data [10.200937444995944]
  LORD is a framework to Leverage Open-set Recognition by exploiting unknown data. We identify three model-agnostic training strategies that exploit background data and applied them to well-established classifiers.
  arXiv  Detail & Related papers  (2023-08-24T06:12:41Z)
• Learning Classifiers of Prototypes and Reciprocal Points for Universal Domain Adaptation [79.62038105814658]
  Universal Domain aims to transfer the knowledge between datasets by handling two shifts: domain-shift and categoryshift. Main challenge is correctly distinguishing the unknown target samples while adapting the distribution of known class knowledge from source to target. Most existing methods approach this problem by first training the target adapted known and then relying on the single threshold to distinguish unknown target samples.
  arXiv  Detail & Related papers  (2022-12-16T09:01:57Z)
• Deepfake Detection via Joint Unsupervised Reconstruction and Supervised Classification [25.84902508816679]
  We introduce a novel approach for deepfake detection, which considers the reconstruction and classification tasks simultaneously. This method shares the information learned by one task with the other, which focuses on a different aspect other existing works rarely consider. Our method achieves state-of-the-art performance on three commonly-used datasets.
  arXiv  Detail & Related papers  (2022-11-24T05:44:26Z)
• Feature Decoupling in Self-supervised Representation Learning for Open Set Recognition [0.0]
  We use a two-stage training strategy for the open set recognition (OSR) problems. In the first stage, we introduce a self-supervised feature decoupling method that finds the content features of the input samples from the known classes. In the second stage, we fine-tune the content features with the class labels. Our experimental results indicate that our proposed self-supervised approach outperforms others in image and malware OSR problems.
  arXiv  Detail & Related papers  (2022-09-28T19:21:53Z)
• Open-set Recognition via Augmentation-based Similarity Learning [11.706887820422002]
  We propose to detect unknowns (or unseen class samples) through learning pairwise similarities. We call our method OPG (Open set recognition based on Pseudo unseen data Generation)
  arXiv  Detail & Related papers  (2022-03-24T17:49:38Z)
• Open-Set Recognition: A Good Closed-Set Classifier is All You Need [146.6814176602689]
  We show that the ability of a classifier to make the 'none-of-above' decision is highly correlated with its accuracy on the closed-set classes. We use this correlation to boost the performance of the cross-entropy OSR 'baseline' by improving its closed-set accuracy. We also construct new benchmarks which better respect the task of detecting semantic novelty.
  arXiv  Detail & Related papers  (2021-10-12T17:58:59Z)
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption [72.35532598131176]
  We propose SCARF, a technique for contrastive learning, where views are formed by corrupting a random subset of features. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders.
  arXiv  Detail & Related papers  (2021-06-29T08:08:33Z)
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data [78.69828864672978]
  A central challenge in training classification models in the real-world federated system is learning with non-IID data. We propose a novel and simple algorithm called Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated ssian mixture model. Experimental results demonstrate that CCVR state-of-the-art performance on popular federated learning benchmarks including CIFAR-10, CIFAR-100, and CINIC-10.
  arXiv  Detail & Related papers  (2021-06-09T12:02:29Z)
• Open-set Recognition based on the Combination of Deep Learning and Ensemble Method for Detecting Unknown Traffic Scenarios [0.9711326718689492]
  This work proposes a combination of Convolutional Neural Networks (CNN) and Random Forest (RF) for open set recognition of traffic scenarios. By inheriting the ensemble nature of RF, the vote pattern of all trees combined with extreme value theory is shown to be well suited for detecting unknown classes.
  arXiv  Detail & Related papers  (2021-05-17T06:48:15Z)
• Conditional Variational Capsule Network for Open Set Recognition [64.18600886936557]
  In open set recognition, a classifier has to detect unknown classes that are not known at training time. Recently proposed Capsule Networks have shown to outperform alternatives in many fields, particularly in image recognition. In our proposal, during training, capsules features of the same known class are encouraged to match a pre-defined gaussian, one for each class.
  arXiv  Detail & Related papers  (2021-04-19T09:39:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.