OpenMix: Exploring Outlier Samples for Misclassification Detection

• URL: http://arxiv.org/abs/2303.17093v1
• Date: Thu, 30 Mar 2023 01:47:23 GMT
• Title: OpenMix: Exploring Outlier Samples for Misclassification Detection
• Authors: Fei Zhu, Zhen Cheng, Xu-Yao Zhang, Cheng-Lin Liu
• Abstract summary: We exploit the easily available outlier samples, i.e., unlabeled samples coming from non-target classes, for helping detect misclassification errors. We propose a novel method called OpenMix, which incorporates open-world knowledge by learning to reject uncertain pseudo-samples generated via outlier transformation.
• Score: 37.43981354073841
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reliable confidence estimation for deep neural classifiers is a challenging yet fundamental requirement in high-stakes applications. Unfortunately, modern deep neural networks are often overconfident for their erroneous predictions. In this work, we exploit the easily available outlier samples, i.e., unlabeled samples coming from non-target classes, for helping detect misclassification errors. Particularly, we find that the well-known Outlier Exposure, which is powerful in detecting out-of-distribution (OOD) samples from unknown classes, does not provide any gain in identifying misclassification errors. Based on these observations, we propose a novel method called OpenMix, which incorporates open-world knowledge by learning to reject uncertain pseudo-samples generated via outlier transformation. OpenMix significantly improves confidence reliability under various scenarios, establishing a strong and unified framework for detecting both misclassified samples from known classes and OOD samples from unknown classes. The code is publicly available at https://github.com/Impression2805/OpenMix.

Related papers

• Reject Threshold Adaptation for Open-Set Model Attribution of Deepfake Audio [37.399771023763485]
  We propose a novel framework for open set model attribution of deepfake audio with rejection threshold adaptation (ReTA) ReTA trains by combining the representation of system fingerprints with labels corresponding to either the target class or a randomly chosen other class label. The reject threshold calculation module utilizes gaussian probability estimation to fit the distributions of matching and non-matching reconstruction errors.
  arXiv  Detail & Related papers  (2024-12-02T12:06:50Z)
• Revisiting Confidence Estimation: Towards Reliable Failure Prediction [53.79160907725975]
  We find a general, widely existing but actually-neglected phenomenon that most confidence estimation methods are harmful for detecting misclassification errors. We propose to enlarge the confidence gap by finding flat minima, which yields state-of-the-art failure prediction performance.
  arXiv  Detail & Related papers  (2024-03-05T11:44:14Z)
• Bidirectional Uncertainty-Based Active Learning for Open Set Annotation [28.624321743629825]
  Active learning in open set scenarios presents a novel challenge of identifying the most valuable examples. Traditional methods prioritize selecting informative examples with low confidence. Recent methods favor the most probable known-class examples. We propose a Bidirectional Uncertainty-based Active Learning framework.
  arXiv  Detail & Related papers  (2024-02-23T08:59:04Z)
• Confidence-aware Training of Smoothed Classifiers for Certified Robustness [75.95332266383417]
  We use "accuracy under Gaussian noise" as an easy-to-compute proxy of adversarial robustness for an input. Our experiments show that the proposed method consistently exhibits improved certified robustness upon state-of-the-art training methods.
  arXiv  Detail & Related papers  (2022-12-18T03:57:12Z)
• Intra-class Adaptive Augmentation with Neighbor Correction for Deep Metric Learning [99.14132861655223]
  We propose a novel intra-class adaptive augmentation (IAA) framework for deep metric learning. We reasonably estimate intra-class variations for every class and generate adaptive synthetic samples to support hard samples mining. Our method significantly improves and outperforms the state-of-the-art methods on retrieval performances by 3%-6%.
  arXiv  Detail & Related papers  (2022-11-29T14:52:38Z)
• Large-Scale Open-Set Classification Protocols for ImageNet [0.0]
  Open-Set Classification (OSC) intends to adapt closed-set classification models to real-world scenarios. We propose three open-set protocols that provide rich datasets of natural images with different levels of similarity between known and unknown classes. We propose a new validation metric that can be employed to assess whether the training of deep learning models addresses both the classification of known samples and the rejection of unknown samples.
  arXiv  Detail & Related papers  (2022-10-13T07:01:34Z)
• Understanding Classifier Mistakes with Generative Models [88.20470690631372]
  Deep neural networks are effective on supervised learning tasks, but have been shown to be brittle. In this paper, we leverage generative models to identify and characterize instances where classifiers fail to generalize. Our approach is agnostic to class labels from the training set which makes it applicable to models trained in a semi-supervised way.
  arXiv  Detail & Related papers  (2020-10-05T22:13:21Z)
• Open Set Recognition with Conditional Probabilistic Generative Models [51.40872765917125]
  We propose Conditional Probabilistic Generative Models (CPGM) for open set recognition. CPGM can detect unknown samples but also classify known classes by forcing different latent features to approximate conditional Gaussian distributions. Experiment results on multiple benchmark datasets reveal that the proposed method significantly outperforms the baselines.
  arXiv  Detail & Related papers  (2020-08-12T06:23:49Z)
• Conditional Gaussian Distribution Learning for Open Set Recognition [10.90687687505665]
  We propose Conditional Gaussian Distribution Learning (CGDL) for open set recognition. In addition to detecting unknown samples, this method can also classify known samples by forcing different latent features to approximate different Gaussian models. Experiments on several standard image reveal that the proposed method significantly outperforms the baseline method and achieves new state-of-the-art results.
  arXiv  Detail & Related papers  (2020-03-19T14:32:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.