Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels

• URL: http://arxiv.org/abs/2012.04193v1
• Date: Tue, 8 Dec 2020 03:37:47 GMT
• Title: Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels
• Authors: Pengfei Chen, Junjie Ye, Guangyong Chen, Jingwei Zhao, Pheng-Ann Heng
• Abstract summary: We show that maximizing training accuracy on sufficiently many noisy samples yields an approximately optimal classifier. For validation, we prove that a noisy validation set is reliable, addressing the critical demand of model selection. We show characterizations of models trained with noisy labels, motivated by our theoretical results, and verify the utility of a noisy validation set.
• Score: 51.66448070984615
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For multi-class classification under class-conditional label noise, we prove that the accuracy metric itself can be robust. We concretize this finding's inspiration in two essential aspects: training and validation, with which we address critical issues in learning with noisy labels. For training, we show that maximizing training accuracy on sufficiently many noisy samples yields an approximately optimal classifier. For validation, we prove that a noisy validation set is reliable, addressing the critical demand of model selection in scenarios like hyperparameter-tuning and early stopping. Previously, model selection using noisy validation samples has not been theoretically justified. We verify our theoretical results and additional claims with extensive experiments. We show characterizations of models trained with noisy labels, motivated by our theoretical results, and verify the utility of a noisy validation set by showing the impressive performance of a framework termed noisy best teacher and student (NTS). Our code is released.

Related papers

• Vision-Language Models are Strong Noisy Label Detectors [76.07846780815794]
  This paper presents a Denoising Fine-Tuning framework, called DeFT, for adapting vision-language models. DeFT utilizes the robust alignment of textual and visual features pre-trained on millions of auxiliary image-text pairs to sieve out noisy labels. Experimental results on seven synthetic and real-world noisy datasets validate the effectiveness of DeFT in both noisy label detection and image classification.
  arXiv  Detail & Related papers  (2024-09-29T12:55:17Z)
• Foster Adaptivity and Balance in Learning with Noisy Labels [26.309508654960354]
  We propose a novel approach named textbfSED to deal with label noise in a textbfSelf-adaptivtextbfE and class-balancetextbfD manner. A mean-teacher model is then employed to correct labels of noisy samples. We additionally propose a self-adaptive and class-balanced sample re-weighting mechanism to assign different weights to detected noisy samples.
  arXiv  Detail & Related papers  (2024-07-03T03:10:24Z)
• Learning with Imbalanced Noisy Data by Preventing Bias in Sample Selection [82.43311784594384]
  Real-world datasets contain not only noisy labels but also class imbalance. We propose a simple yet effective method to address noisy labels in imbalanced datasets.
  arXiv  Detail & Related papers  (2024-02-17T10:34:53Z)
• Learning to Detect Noisy Labels Using Model-Based Features [16.681748918518075]
  We propose Selection-Enhanced Noisy label Training (SENT) SENT does not rely on meta learning while having the flexibility of being data-driven. It improves performance over strong baselines under the settings of self-training and label corruption.
  arXiv  Detail & Related papers  (2022-12-28T10:12:13Z)
• Confidence-aware Training of Smoothed Classifiers for Certified Robustness [75.95332266383417]
  We use "accuracy under Gaussian noise" as an easy-to-compute proxy of adversarial robustness for an input. Our experiments show that the proposed method consistently exhibits improved certified robustness upon state-of-the-art training methods.
  arXiv  Detail & Related papers  (2022-12-18T03:57:12Z)
• Label Noise-Robust Learning using a Confidence-Based Sieving Strategy [15.997774467236352]
  In learning tasks with label noise, improving model robustness against overfitting is a pivotal challenge. Identifying the samples with noisy labels and preventing the model from learning them is a promising approach to address this challenge. We propose a novel discriminator metric called confidence error and a sieving strategy called CONFES to differentiate between the clean and noisy samples effectively.
  arXiv  Detail & Related papers  (2022-10-11T10:47:28Z)
• Centrality and Consistency: Two-Stage Clean Samples Identification for Learning with Instance-Dependent Noisy Labels [87.48541631675889]
  We propose a two-stage clean samples identification method. First, we employ a class-level feature clustering procedure for the early identification of clean samples. Second, for the remaining clean samples that are close to the ground truth class boundary, we propose a novel consistency-based classification method.
  arXiv  Detail & Related papers  (2022-07-29T04:54:57Z)
• Constrained Instance and Class Reweighting for Robust Learning under Label Noise [20.30785186456126]
  We propose a principled approach for tackling label noise with the aim of assigning importance weights to individual instances and class labels. Our method works by formulating a class of constrained optimization problems that yield simple closed form updates for these importance weights. We evaluate our method on several benchmark datasets and observe considerable performance gains in the presence of label noise.
  arXiv  Detail & Related papers  (2021-11-09T21:37:53Z)
• Noisy Self-Knowledge Distillation for Text Summarization [83.49809205891496]
  We apply self-knowledge distillation to text summarization which we argue can alleviate problems with maximum-likelihood training. Our student summarization model is trained with guidance from a teacher which generates smoothed labels to help regularize training. We demonstrate experimentally on three benchmarks that our framework boosts the performance of both pretrained and non-pretrained summarizers.
  arXiv  Detail & Related papers  (2020-09-15T12:53:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.