Detecting Novelties with Empty Classes

• URL: http://arxiv.org/abs/2305.00983v1
• Date: Sun, 30 Apr 2023 19:52:47 GMT
• Title: Detecting Novelties with Empty Classes
• Authors: Svenja Uhlemeyer, Julian Lienen, Eyke H\"ullermeier and Hanno Gottschalk
• Abstract summary: We build upon anomaly detection to retrieve out-of-distribution (OoD) data as candidates for new classes. We introduce two loss functions, which 1) entice the DNN to assign OoD samples to the empty classes and 2) to minimize the inner-class feature distances between them.
• Score: 6.953730499849023
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For open world applications, deep neural networks (DNNs) need to be aware of previously unseen data and adaptable to evolving environments. Furthermore, it is desirable to detect and learn novel classes which are not included in the DNNs underlying set of semantic classes in an unsupervised fashion. The method proposed in this article builds upon anomaly detection to retrieve out-of-distribution (OoD) data as candidates for new classes. We thereafter extend the DNN by $k$ empty classes and fine-tune it on the OoD data samples. To this end, we introduce two loss functions, which 1) entice the DNN to assign OoD samples to the empty classes and 2) to minimize the inner-class feature distances between them. Thus, instead of ground truth which contains labels for the different novel classes, the DNN obtains a single OoD label together with a distance matrix, which is computed in advance. We perform several experiments for image classification and semantic segmentation, which demonstrate that a DNN can extend its own semantic space by multiple classes without having access to ground truth.

Related papers

• Early-Exit with Class Exclusion for Efficient Inference of Neural Networks [4.180653524441411]
  We propose a class-based early-exit for dynamic inference in deep neural networks (DNNs) We take advantage of the learned features in these layers to exclude as many irrelevant classes as possible. Experimental results demonstrate the computational cost of DNNs in inference can be reduced significantly.
  arXiv  Detail & Related papers  (2023-09-23T18:12:27Z)
• Upcycling Models under Domain and Category Shift [95.22147885947732]
  We introduce an innovative global and local clustering learning technique (GLC) We design a novel, adaptive one-vs-all global clustering algorithm to achieve the distinction across different target classes. Remarkably, in the most challenging open-partial-set DA scenario, GLC outperforms UMAD by 14.8% on the VisDA benchmark.
  arXiv  Detail & Related papers  (2023-03-13T13:44:04Z)
• Rethinking Nearest Neighbors for Visual Classification [56.00783095670361]
  k-NN is a lazy learning method that aggregates the distance between the test image and top-k neighbors in a training set. We adopt k-NN with pre-trained visual representations produced by either supervised or self-supervised methods in two steps. Via extensive experiments on a wide range of classification tasks, our study reveals the generality and flexibility of k-NN integration.
  arXiv  Detail & Related papers  (2021-12-15T20:15:01Z)
• Hybrid Graph Neural Networks for Few-Shot Learning [85.93495480949079]
  Graph neural networks (GNNs) have been used to tackle the few-shot learning problem. Under the inductive setting, existing GNN based methods are less competitive. We propose a novel hybrid GNN model consisting of two GNNs, an instance GNN and a prototype GNN.
  arXiv  Detail & Related papers  (2021-12-13T10:20:15Z)
• Evaluating Deep Neural Network Ensembles by Majority Voting cum Meta-Learning scheme [3.351714665243138]
  We propose an ensemble of seven independent Deep Neural Networks (DNNs) for a new data instance. One-seventh of the data is deleted and replenished by bootstrap sampling from the remaining samples. All the algorithms in this paper have been tested on five benchmark datasets.
  arXiv  Detail & Related papers  (2021-05-09T03:10:56Z)
• OVANet: One-vs-All Network for Universal Domain Adaptation [78.86047802107025]
  Existing methods manually set a threshold to reject unknown samples based on validation or a pre-defined ratio of unknown samples. We propose a method to learn the threshold using source samples and to adapt it to the target domain. Our idea is that a minimum inter-class distance in the source domain should be a good threshold to decide between known or unknown in the target.
  arXiv  Detail & Related papers  (2021-04-07T18:36:31Z)
• Meta Discovery: Learning to Discover Novel Classes given Very Limited Data [59.90813997957849]
  In this paper, we analyze and improve L2DNC by linking it to meta-learning. L2DNC is not only theoretically solvable, but also can be empirically solved by meta-learning algorithms slightly modified to fit our proposed framework.
  arXiv  Detail & Related papers  (2021-02-08T04:53:14Z)
• Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentation [7.305019142196585]
  "Out-of-distribution" (OoD) samples are crucial for many applications such as automated driving. A natural baseline approach to OoD detection is to threshold on the pixel-wise softmax entropy. We present a two-step procedure that significantly improves that approach.
  arXiv  Detail & Related papers  (2020-12-09T11:01:06Z)
• Bridging the Theoretical Bound and Deep Algorithms for Open Set Domain Adaptation [40.95099721257058]
  In the unsupervised open set domain adaptation (UOSDA), the target domain contains unknown classes that are not observed in the source domain. We propose a new upper bound of target-domain risk for UOSDA, which includes four terms: source-domain risk, $epsilon$-open set difference ($Delta_epsilon$), a distributional discrepancy between domains, and a constant. Specifically, source-domain risk and $Delta_epsilon$ are minimized by gradient descent, and the distributional discrepancy is minimized via a novel open-set conditional adversarial training strategy.
  arXiv  Detail & Related papers  (2020-06-23T14:01:06Z)
• One Versus all for deep Neural Network Incertitude (OVNNI) quantification [12.734278426543332]
  We propose a new technique to quantify the epistemic uncertainty of data easily. This method consists in mixing the predictions of an ensemble of DNNs trained to classify One class vs All the other classes (OVA) with predictions from a standard DNN trained to perform All vs All (AVA) classification.
  arXiv  Detail & Related papers  (2020-06-01T14:06:12Z)
• L-Vector: Neural Label Embedding for Domain Adaptation [62.112885747045766]
  We propose a novel neural label embedding (NLE) scheme for the domain adaptation of a deep neural network (DNN) acoustic model with unpaired data samples. NLE achieves up to 14.1% relative word error rate reduction over direct re-training with one-hot labels.
  arXiv  Detail & Related papers  (2020-04-25T06:40:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.