Teacher-Explorer-Student Learning: A Novel Learning Method for Open Set
Recognition
- URL: http://arxiv.org/abs/2103.12871v1
- Date: Tue, 23 Mar 2021 22:32:32 GMT
- Title: Teacher-Explorer-Student Learning: A Novel Learning Method for Open Set
Recognition
- Authors: Jaeyeon Jang and Chang Ouk Kim
- Abstract summary: Teacher-explorer-student (T/E/S) learning aims to reject unknown samples while minimizing the loss of classification performance on known samples.
In this novel learning method, overgeneralization of deep learning classifiers is significantly reduced by exploring various possibilities of unknowns.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: If an unknown example that is not seen during training appears, most
recognition systems usually produce overgeneralized results and determine that
the example belongs to one of the known classes. To address this problem,
teacher-explorer-student (T/E/S) learning, which adopts the concept of open set
recognition (OSR) that aims to reject unknown samples while minimizing the loss
of classification performance on known samples, is proposed in this study. In
this novel learning method, overgeneralization of deep learning classifiers is
significantly reduced by exploring various possibilities of unknowns. Here, the
teacher network extracts some hints about unknowns by distilling the pretrained
knowledge about knowns and delivers this distilled knowledge to the student.
After learning the distilled knowledge, the student network shares the learned
information with the explorer network. Then, the explorer network shares its
exploration results by generating unknown-like samples and feeding the samples
to the student network. By repeating this alternating learning process, the
student network experiences a variety of synthetic unknowns, reducing
overgeneralization. Extensive experiments were conducted, and the experimental
results showed that each component proposed in this paper significantly
contributes to the improvement in OSR performance. As a result, the proposed
T/E/S learning method outperformed current state-of-the-art methods.
Related papers
- OpenIncrement: A Unified Framework for Open Set Recognition and Deep
Class-Incremental Learning [4.278434830731282]
We introduce a deep class-incremental learning framework integrated with open set recognition.
Our approach refines class-incrementally learned features to adapt them for distance-based open set recognition.
Experimental results validate that our method outperforms state-of-the-art incremental learning techniques.
arXiv Detail & Related papers (2023-10-05T19:08:08Z) - Learning to Retain while Acquiring: Combating Distribution-Shift in
Adversarial Data-Free Knowledge Distillation [31.294947552032088]
Data-free Knowledge Distillation (DFKD) has gained popularity recently, with the fundamental idea of carrying out knowledge transfer from a Teacher to a Student neural network in the absence of training data.
We propose a meta-learning inspired framework by treating the task of Knowledge-Acquisition (learning from newly generated samples) and Knowledge-Retention (retaining knowledge on previously met samples) as meta-train and meta-test.
arXiv Detail & Related papers (2023-02-28T03:50:56Z) - The Devil is in the Wrongly-classified Samples: Towards Unified Open-set
Recognition [61.28722817272917]
Open-set Recognition (OSR) aims to identify test samples whose classes are not seen during the training process.
Recently, Unified Open-set Recognition (UOSR) has been proposed to reject not only unknown samples but also known but wrongly classified samples.
arXiv Detail & Related papers (2023-02-08T11:34:04Z) - Learning with Recoverable Forgetting [77.56338597012927]
Learning wIth Recoverable Forgetting explicitly handles the task- or sample-specific knowledge removal and recovery.
Specifically, LIRF brings in two innovative schemes, namely knowledge deposit and withdrawal.
We conduct experiments on several datasets, and demonstrate that the proposed LIRF strategy yields encouraging results with gratifying generalization capability.
arXiv Detail & Related papers (2022-07-17T16:42:31Z) - Class-Specific Semantic Reconstruction for Open Set Recognition [101.24781422480406]
Open set recognition enables deep neural networks (DNNs) to identify samples of unknown classes.
We propose a novel method, called Class-Specific Semantic Reconstruction (CSSR), that integrates the power of auto-encoder (AE) and prototype learning.
Results of experiments conducted on multiple datasets show that the proposed method achieves outstanding performance in both close and open set recognition.
arXiv Detail & Related papers (2022-07-05T16:25:34Z) - A Unified End-to-End Retriever-Reader Framework for Knowledge-based VQA [67.75989848202343]
This paper presents a unified end-to-end retriever-reader framework towards knowledge-based VQA.
We shed light on the multi-modal implicit knowledge from vision-language pre-training models to mine its potential in knowledge reasoning.
Our scheme is able to not only provide guidance for knowledge retrieval, but also drop these instances potentially error-prone towards question answering.
arXiv Detail & Related papers (2022-06-30T02:35:04Z) - Synthetic Unknown Class Learning for Learning Unknowns [0.0]
This paper proposes a novel synthetic unknown class learning method.
It generates unknown-like samples while maintaining diversity between the generated samples and learns these samples.
Experiments on several benchmark datasets show that the proposed method significantly outperforms other state-of-the-art approaches.
arXiv Detail & Related papers (2021-11-15T19:46:41Z) - Collective Decision of One-vs-Rest Networks for Open Set Recognition [0.0]
We propose a simple open set recognition (OSR) method based on the intuition that OSR performance can be maximized by setting strict and sophisticated decision boundaries.
The proposed method performed significantly better than the state-of-the-art methods by effectively reducing overgeneralization.
arXiv Detail & Related papers (2021-03-18T13:06:46Z) - Learning Open Set Network with Discriminative Reciprocal Points [70.28322390023546]
Open set recognition aims to simultaneously classify samples from predefined classes and identify the rest as 'unknown'
In this paper, we propose a new concept, Reciprocal Point, which is the potential representation of the extra-class space corresponding to each known category.
Based on the bounded space constructed by reciprocal points, the risk of unknown is reduced through multi-category interaction.
arXiv Detail & Related papers (2020-10-31T03:20:31Z) - Conditional Gaussian Distribution Learning for Open Set Recognition [10.90687687505665]
We propose Conditional Gaussian Distribution Learning (CGDL) for open set recognition.
In addition to detecting unknown samples, this method can also classify known samples by forcing different latent features to approximate different Gaussian models.
Experiments on several standard image reveal that the proposed method significantly outperforms the baseline method and achieves new state-of-the-art results.
arXiv Detail & Related papers (2020-03-19T14:32:08Z) - Learning From Multiple Experts: Self-paced Knowledge Distillation for
Long-tailed Classification [106.08067870620218]
We propose a self-paced knowledge distillation framework, termed Learning From Multiple Experts (LFME)
We refer to these models as 'Experts', and the proposed LFME framework aggregates the knowledge from multiple 'Experts' to learn a unified student model.
We conduct extensive experiments and demonstrate that our method is able to achieve superior performances compared to state-of-the-art methods.
arXiv Detail & Related papers (2020-01-06T12:57:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.