Stacked Confusion Reject Plots (SCORE)

• URL: http://arxiv.org/abs/2406.17346v1
• Date: Tue, 25 Jun 2024 07:59:29 GMT
• Title: Stacked Confusion Reject Plots (SCORE)
• Authors: Stephan Hasler, Lydia Fischer,
• Abstract summary: We argue that common reject curves are too abstract and hard to interpret by non-experts. We propose Stacked Confusion Reject Plots (SCORE) that offer a more intuitive understanding of the used data.
• Score: 0.7673339435080445
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning is more and more applied in critical application areas like health and driver assistance. To minimize the risk of wrong decisions, in such applications it is necessary to consider the certainty of a classification to reject uncertain samples. An established tool for this are reject curves that visualize the trade-off between the number of rejected samples and classification performance metrics. We argue that common reject curves are too abstract and hard to interpret by non-experts. We propose Stacked Confusion Reject Plots (SCORE) that offer a more intuitive understanding of the used data and the classifier's behavior. We present example plots on artificial Gaussian data to document the different options of SCORE and provide the code as a Python package.

Related papers

• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions. We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance. Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• Logic-based Explanations for Linear Support Vector Classifiers with Reject Option [0.0]
  Support Vector (SVC) is a well-known Machine Learning (ML) model for linear classification problems. We propose a logic-based approach with formal guarantees on the correctness and minimality of explanations for linear SVCs with reject option.
  arXiv  Detail & Related papers  (2024-03-24T15:14:44Z)
• Dirichlet-Based Prediction Calibration for Learning with Noisy Labels [40.78497779769083]
  Learning with noisy labels can significantly hinder the generalization performance of deep neural networks (DNNs) Existing approaches address this issue through loss correction or example selection methods. We propose the textitDirichlet-based Prediction (DPC) method as a solution.
  arXiv  Detail & Related papers  (2024-01-13T12:33:04Z)
• Virtual Category Learning: A Semi-Supervised Learning Method for Dense Prediction with Extremely Limited Labels [63.16824565919966]
  This paper proposes to use confusing samples proactively without label correction. A Virtual Category (VC) is assigned to each confusing sample in such a way that it can safely contribute to the model optimisation. Our intriguing findings highlight the usage of VC learning in dense vision tasks.
  arXiv  Detail & Related papers  (2023-12-02T16:23:52Z)
• XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners [71.8257151788923]
  We propose a novel Explainable Active Learning framework (XAL) for low-resource text classification. XAL encourages classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations. Experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines.
  arXiv  Detail & Related papers  (2023-10-09T08:07:04Z)
• Adaptive Negative Evidential Deep Learning for Open-set Semi-supervised Learning [69.81438976273866]
  Open-set semi-supervised learning (Open-set SSL) considers a more practical scenario, where unlabeled data and test data contain new categories (outliers) not observed in labeled data (inliers) We introduce evidential deep learning (EDL) as an outlier detector to quantify different types of uncertainty, and design different uncertainty metrics for self-training and inference. We propose a novel adaptive negative optimization strategy, making EDL more tailored to the unlabeled dataset containing both inliers and outliers.
  arXiv  Detail & Related papers  (2023-03-21T09:07:15Z)
• Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
  Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data. We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
  arXiv  Detail & Related papers  (2023-03-06T14:49:59Z)
• Learning When to Say "I Don't Know" [0.5505634045241288]
  We propose a new Reject Option Classification technique to identify and remove regions of uncertainty in the decision space. We consider an alternative formulation by instead analyzing the complementary reject region and employing a validation set to learn per-class softmax thresholds. We provide results showing the benefits of the proposed method over na"ively thresholding/uncalibrated softmax scores with 2-D points, imagery, and text classification datasets.
  arXiv  Detail & Related papers  (2022-09-11T21:50:03Z)
• Semi-supervised Object Detection via Virtual Category Learning [68.26956850996976]
  This paper proposes to use confusing samples proactively without label correction. Specifically, a virtual category (VC) is assigned to each confusing sample. It is attributed to specifying the embedding distance between the training sample and the virtual category.
  arXiv  Detail & Related papers  (2022-07-07T16:59:53Z)
• Autoencoder Attractors for Uncertainty Estimation [13.618797548020462]
  We propose a novel approach for uncertainty estimation based on autoencoder models. We evaluate our approach on several dataset combinations as well as on an industrial application for occupant classification in the vehicle interior.
  arXiv  Detail & Related papers  (2022-04-01T12:10:06Z)
• RISAN: Robust Instance Specific Abstention Network [13.303728978965072]
  We propose deep architectures for learning instance abstain (reject option) binary classifiers. The proposed approach uses double sigmoid loss function as described by Kulin Shah and Naresh Manwani. We observe that the proposed approach not only performs comparable to the state-of-the-art approaches, it is also robust against label noise.
  arXiv  Detail & Related papers  (2021-07-07T09:14:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.