Asymmetric error control under imperfect supervision: a label-noise-adjusted Neyman-Pearson umbrella algorithm

• URL: http://arxiv.org/abs/2112.00314v1
• Date: Wed, 1 Dec 2021 07:10:19 GMT
• Title: Asymmetric error control under imperfect supervision: a label-noise-adjusted Neyman-Pearson umbrella algorithm
• Abstract summary: Neyman-Pearson (NP) classification paradigm constrains the more severe type of error under a preferred level. We propose the first theory-backed algorithm that adapts most state-of-the-art classification methods to the training label noise under the NP paradigm.
• Score: 9.680436421852768
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Label noise in data has long been an important problem in supervised learning applications as it affects the effectiveness of many widely used classification methods. Recently, important real-world applications, such as medical diagnosis and cybersecurity, have generated renewed interest in the Neyman-Pearson (NP) classification paradigm, which constrains the more severe type of error (e.g., the type I error) under a preferred level while minimizing the other (e.g., the type II error). However, there has been little research on the NP paradigm under label noise. It is somewhat surprising that even when common NP classifiers ignore the label noise in the training stage, they are still able to control the type I error with high probability. However, the price they pay is excessive conservativeness of the type I error and hence a significant drop in power (i.e., $1 - $ type II error). Assuming that domain experts provide lower bounds on the corruption severity, we propose the first theory-backed algorithm that adapts most state-of-the-art classification methods to the training label noise under the NP paradigm. The resulting classifiers not only control the type I error with high probability under the desired level but also improve power.

Related papers

• Neyman-Pearson multiclass classification under label noise via empirical likelihood [12.527550498929713]
  In many classification problems, the costs of misclassifying observations from different classes can be highly unequal. The Neyman-Pearson multiclass classification (NPMC) framework addresses this issue by minimizing a weighted misclassification risk while imposing upper bounds on class-specific error probabilities.<n>We study the NPMC problem when only noisy labels are available in the training data. We propose an empirical likelihood (EL)-based method that relates distributions of noisy and true labels through an exponential tilting density ratio model.
  arXiv  Detail & Related papers  (2026-03-23T06:39:09Z)
• Sharpness-aware Dynamic Anchor Selection for Generalized Category Discovery [61.694524826522205]
  Given some labeled data of known classes, GCD aims to cluster unlabeled data that contain both known and unknown classes.<n>Large pre-trained models have a preference for some specific visual patterns, resulting in encoding spurious correlation for unlabeled data.<n>We propose a novel method, which contains two modules: Loss Sharpness Penalty (LSP) and Dynamic Anchor Selection (DAS)
  arXiv  Detail & Related papers  (2025-12-15T02:24:06Z)
• The Exploration of Error Bounds in Classification with Noisy Labels [7.657250843344973]
  In this paper, we focus on the error bounds of excess risks for classification problems with noisy labels within deep learning frameworks.<n>We derive error bounds for the excess risk, decomposing it into statistical error and approximation error.<n>Under the low-dimensional manifold hypothesis, we further refine the approximation error to mitigate the impact of high-dimensional input spaces.
  arXiv  Detail & Related papers  (2025-01-25T10:06:50Z)
• Generating Unbiased Pseudo-labels via a Theoretically Guaranteed Chebyshev Constraint to Unify Semi-supervised Classification and Regression [57.17120203327993]
  threshold-to-pseudo label process (T2L) in classification uses confidence to determine the quality of label. In nature, regression also requires unbiased methods to generate high-quality labels. We propose a theoretically guaranteed constraint for generating unbiased labels based on Chebyshev's inequality.
  arXiv  Detail & Related papers  (2023-11-03T08:39:35Z)
• Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels [61.97359362447732]
  Learning from noisy labels is an important and long-standing problem in machine learning for real applications. In this paper, we reformulate the label-noise problem from a generative-model perspective. Our model achieves new state-of-the-art (SOTA) results on all the standard real-world benchmark datasets.
  arXiv  Detail & Related papers  (2023-05-31T03:01:36Z)
• Test-Time Amendment with a Coarse Classifier for Fine-Grained Classification [10.719054378755981]
  We present a novel approach for Post-Hoc Correction called Hierarchical Ensembles (HiE) HiE utilizes label hierarchy to improve the performance of fine-grained classification at test-time using the coarse-grained predictions. Our approach brings notable gains in top-1 accuracy while significantly decreasing the severity of mistakes as training data decreases for the fine-grained classes.
  arXiv  Detail & Related papers  (2023-02-01T10:55:27Z)
• Hierarchical Neyman-Pearson Classification for Prioritizing Severe Disease Categories in COVID-19 Patient Data [14.74172876249796]
  COVID-19 has a spectrum of disease severity, ranging from asymptomatic to requiring hospitalization. It is beneficial to prioritize the identification of more severe classes and control the "under-classification" errors. Here, we propose a hierarchical NP (H-NP) framework and an umbrella algorithm that controls the under-classification errors with high probability.
  arXiv  Detail & Related papers  (2022-10-01T06:11:30Z)
• Do We Need to Penalize Variance of Losses for Learning with Label Noise? [91.38888889609002]
  We find that the variance should be increased for the problem of learning with noisy labels. By exploiting the label noise transition matrix, regularizers can be easily designed to reduce the variance of losses. Empirically, the proposed method by increasing the variance of losses significantly improves the generalization ability of baselines on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-01-30T06:19:08Z)
• Prototypical Classifier for Robust Class-Imbalanced Learning [64.96088324684683]
  We propose textitPrototypical, which does not require fitting additional parameters given the embedding network. Prototypical produces balanced and comparable predictions for all classes even though the training set is class-imbalanced. We test our method on CIFAR-10LT, CIFAR-100LT and Webvision datasets, observing that Prototypical obtains substaintial improvements compared with state of the arts.
  arXiv  Detail & Related papers  (2021-10-22T01:55:01Z)
• CCMN: A General Framework for Learning with Class-Conditional Multi-Label Noise [40.46921277898713]
  Class-conditional noise commonly exists in machine learning tasks, where the class label is corrupted with a probability depending on its ground-truth. In this paper, we formalize this problem as a general framework of learning with Class-Conditional Multi-label Noise ( CCMN for short) We establish two unbiased estimators with error bounds for solving the CCMN problems, and prove that they are consistent with commonly used multi-label loss functions.
  arXiv  Detail & Related papers  (2021-05-16T03:24:15Z)
• Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model [80.91927573604438]
  This paper proposes a simple yet universal probabilistic model, which explicitly relates noisy labels to their instances. Experiments on datasets with both synthetic and real-world label noise verify that the proposed method yields significant improvements on robustness.
  arXiv  Detail & Related papers  (2021-01-14T05:43:51Z)
• Bridging Cost-sensitive and Neyman-Pearson Paradigms for Asymmetric Binary Classification [12.083020037901651]
  We study the methodological connections between the cost-sensitive and Neyman-Pearson paradigms. We develop the TUBE-CS algorithm to bridge the two paradigms.
  arXiv  Detail & Related papers  (2020-12-29T21:40:52Z)
• A Second-Order Approach to Learning with Instance-Dependent Label Noise [58.555527517928596]
  The presence of label noise often misleads the training of deep neural networks. We show that the errors in human-annotated labels are more likely to be dependent on the difficulty levels of tasks.
  arXiv  Detail & Related papers  (2020-12-22T06:36:58Z)
• Error-Bounded Correction of Noisy Labels [17.510654621245656]
  We show that the prediction of a noisy classifier can indeed be a good indicator of whether the label of a training data is clean. Based on the theoretical result, we propose a novel algorithm that corrects the labels based on the noisy classifier prediction. We incorporate our label correction algorithm into the training of deep neural networks and train models that achieve superior testing performance on multiple public datasets.
  arXiv  Detail & Related papers  (2020-11-19T19:23:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.