Correlated Input-Dependent Label Noise in Large-Scale Image Classification

• URL: http://arxiv.org/abs/2105.10305v1
• Date: Wed, 19 May 2021 17:30:59 GMT
• Title: Correlated Input-Dependent Label Noise in Large-Scale Image Classification
• Authors: Mark Collier, Basil Mustafa, Efi Kokiopoulou, Rodolphe Jenatton and Jesse Berent
• Abstract summary: We take a principled probabilistic approach to modelling input-dependent, also known as heteroscedastic, label noise in datasets. We demonstrate that the learned covariance structure captures known sources of label noise between semantically similar and co-occurring classes. We set a new state-of-the-art result on WebVision 1.0 with 76.6% top-1 accuracy.
• Score: 4.979361059762468
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large scale image classification datasets often contain noisy labels. We take a principled probabilistic approach to modelling input-dependent, also known as heteroscedastic, label noise in these datasets. We place a multivariate Normal distributed latent variable on the final hidden layer of a neural network classifier. The covariance matrix of this latent variable, models the aleatoric uncertainty due to label noise. We demonstrate that the learned covariance structure captures known sources of label noise between semantically similar and co-occurring classes. Compared to standard neural network training and other baselines, we show significantly improved accuracy on Imagenet ILSVRC 2012 79.3% (+2.6%), Imagenet-21k 47.0% (+1.1%) and JFT 64.7% (+1.6%). We set a new state-of-the-art result on WebVision 1.0 with 76.6% top-1 accuracy. These datasets range from over 1M to over 300M training examples and from 1k classes to more than 21k classes. Our method is simple to use, and we provide an implementation that is a drop-in replacement for the final fully-connected layer in a deep classifier.

Related papers

• Semi-Supervised Learning with Pseudo-Negative Labels for Image Classification [14.100569951592417]
  We propose a mutual learning framework based on pseudo-negative labels. By reducing the prediction probability on pseudo-negative labels, the dual model can improve its prediction ability. Our framework achieves state-of-the-art results on several main benchmarks.
  arXiv  Detail & Related papers  (2023-01-10T14:15:17Z)
• Invariant Learning via Diffusion Dreamed Distribution Shifts [121.71383835729848]
  We propose a dataset called Diffusion Dreamed Distribution Shifts (D3S) D3S consists of synthetic images generated through StableDiffusion using text prompts and image guides obtained by pasting a sample foreground image onto a background template image. Due to the incredible photorealism of the diffusion model, our images are much closer to natural images than previous synthetic datasets.
  arXiv  Detail & Related papers  (2022-11-18T17:07:43Z)
• Learning with Neighbor Consistency for Noisy Labels [69.83857578836769]
  We present a method for learning from noisy labels that leverages similarities between training examples in feature space. We evaluate our method on datasets evaluating both synthetic (CIFAR-10, CIFAR-100) and realistic (mini-WebVision, Clothing1M, mini-ImageNet-Red) noise.
  arXiv  Detail & Related papers  (2022-02-04T15:46:27Z)
• Improved Regularization and Robustness for Fine-tuning in Neural Networks [5.626364462708321]
  A widely used algorithm for transfer learning is fine-tuning, where a pre-trained model is fine-tuned on a target task with a small amount of labeled data. We propose regularized self-labeling -- the generalization between regularization and self-labeling. Our approach improves baseline methods by 1.76% (on average) for seven image classification tasks and 0.75% for a few-shot classification task.
  arXiv  Detail & Related papers  (2021-11-08T15:39:44Z)
• Instance-based Label Smoothing For Better Calibrated Classification Networks [3.388509725285237]
  Label smoothing is widely used in deep neural networks for multi-class classification. We take inspiration from both label smoothing and self-distillation. We propose two novel instance-based label smoothing approaches.
  arXiv  Detail & Related papers  (2021-10-11T15:33:23Z)
• PLM: Partial Label Masking for Imbalanced Multi-label Classification [59.68444804243782]
  Neural networks trained on real-world datasets with long-tailed label distributions are biased towards frequent classes and perform poorly on infrequent classes. We propose a method, Partial Label Masking (PLM), which utilizes this ratio during training. Our method achieves strong performance when compared to existing methods on both multi-label (MultiMNIST and MSCOCO) and single-label (imbalanced CIFAR-10 and CIFAR-100) image classification datasets.
  arXiv  Detail & Related papers  (2021-05-22T18:07:56Z)
• Attention-Aware Noisy Label Learning for Image Classification [97.26664962498887]
  Deep convolutional neural networks (CNNs) learned on large-scale labeled samples have achieved remarkable progress in computer vision. The cheapest way to obtain a large body of labeled visual data is to crawl from websites with user-supplied labels, such as Flickr. This paper proposes the attention-aware noisy label learning approach to improve the discriminative capability of the network trained on datasets with potential label noise.
  arXiv  Detail & Related papers  (2020-09-30T15:45:36Z)
• SCAN: Learning to Classify Images without Labels [73.69513783788622]
  We advocate a two-step approach where feature learning and clustering are decoupled. A self-supervised task from representation learning is employed to obtain semantically meaningful features. We obtain promising results on ImageNet, and outperform several semi-supervised learning methods in the low-data regime.
  arXiv  Detail & Related papers  (2020-05-25T18:12:33Z)
• Learning from Noisy Labels with Noise Modeling Network [7.523041606515877]
  Noise Modeling Network (NMN) follows our convolutional neural network (CNN) and integrates with it. NMN learns the distribution of noise patterns directly from the noisy data. We show that the integrated NMN/CNN learning system consistently improves the classification performance.
  arXiv  Detail & Related papers  (2020-05-01T20:32:22Z)
• Multi-Class Classification from Noisy-Similarity-Labeled Data [98.13491369929798]
  We propose a method for learning from only noisy-similarity-labeled data. We use a noise transition matrix to bridge the class-posterior probability between clean and noisy data. We build a novel learning system which can assign noise-free class labels for instances.
  arXiv  Detail & Related papers  (2020-02-16T05:10:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.