Adaptive Label Smoothing

• URL: http://arxiv.org/abs/2009.06432v2
• Date: Mon, 7 Dec 2020 23:19:08 GMT
• Title: Adaptive Label Smoothing
• Authors: Ujwal Krothapalli and A. Lynn Abbott
• Abstract summary: We present a novel approach to classification that combines the ideas of objectness and label smoothing during training. We show extensive results using ImageNet to demonstrate that CNNs trained using adaptive label smoothing are much less likely to be overconfident in their predictions.
• Score: 1.3198689566654107
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper concerns the use of objectness measures to improve the calibration performance of Convolutional Neural Networks (CNNs). CNNs have proven to be very good classifiers and generally localize objects well; however, the loss functions typically used to train classification CNNs do not penalize inability to localize an object, nor do they take into account an object's relative size in the given image. During training on ImageNet-1K almost all approaches use random crops on the images and this transformation sometimes provides the CNN with background only samples. This causes the classifiers to depend on context. Context dependence is harmful for safety-critical applications. We present a novel approach to classification that combines the ideas of objectness and label smoothing during training. Unlike previous methods, we compute a smoothing factor that is \emph{adaptive} based on relative object size within an image. This causes our approach to produce confidences that are grounded in the size of the object being classified instead of relying on context to make the correct predictions. We present extensive results using ImageNet to demonstrate that CNNs trained using adaptive label smoothing are much less likely to be overconfident in their predictions. We show qualitative results using class activation maps and quantitative results using classification and transfer learning tasks. Our approach is able to produce an order of magnitude reduction in confidence when predicting on context only images when compared to baselines. Using transfer learning, we gain 2.1mAP on MS COCO compared to the hard label approach.

Related papers

• Fine-grained Recognition with Learnable Semantic Data Augmentation [68.48892326854494]
  Fine-grained image recognition is a longstanding computer vision challenge. We propose diversifying the training data at the feature-level to alleviate the discriminative region loss problem. Our method significantly improves the generalization performance on several popular classification networks.
  arXiv  Detail & Related papers  (2023-09-01T11:15:50Z)
• Facilitated machine learning for image-based fruit quality assessment in developing countries [68.8204255655161]
  Automated image classification is a common task for supervised machine learning in food science. We propose an alternative method based on pre-trained vision transformers (ViTs) It can be easily implemented with limited resources on a standard device.
  arXiv  Detail & Related papers  (2022-07-10T19:52:20Z)
• Do We Really Need a Learnable Classifier at the End of Deep Neural Network? [118.18554882199676]
  We study the potential of learning a neural network for classification with the classifier randomly as an ETF and fixed during training. Our experimental results show that our method is able to achieve similar performances on image classification for balanced datasets.
  arXiv  Detail & Related papers  (2022-03-17T04:34:28Z)
• TransformNet: Self-supervised representation learning through predicting geometric transformations [0.8098097078441623]
  We describe the unsupervised semantic feature learning approach for recognition of the geometric transformation applied to the input data. The basic concept of our approach is that if someone is unaware of the objects in the images, he/she would not be able to quantitatively predict the geometric transformation that was applied to them.
  arXiv  Detail & Related papers  (2022-02-08T22:41:01Z)
• Image Quality Assessment using Contrastive Learning [50.265638572116984]
  We train a deep Convolutional Neural Network (CNN) using a contrastive pairwise objective to solve the auxiliary problem. We show through extensive experiments that CONTRIQUE achieves competitive performance when compared to state-of-the-art NR image quality models. Our results suggest that powerful quality representations with perceptual relevance can be obtained without requiring large labeled subjective image quality datasets.
  arXiv  Detail & Related papers  (2021-10-25T21:01:00Z)
• Exploring the Interchangeability of CNN Embedding Spaces [0.5735035463793008]
  We map between 10 image-classification CNNs and between 4 facial-recognition CNNs. For CNNs trained to the same classes and sharing a common backend-logit architecture, a linear-mapping may always be calculated directly from the backend layer weights. The implications are far-reaching, suggesting an underlying commonality between representations learned by networks designed and trained for a common task.
  arXiv  Detail & Related papers  (2020-10-05T20:32:40Z)
• Learning CNN filters from user-drawn image markers for coconut-tree image classification [78.42152902652215]
  We present a method that needs a minimal set of user-selected images to train the CNN's feature extractor. The method learns the filters of each convolutional layer from user-drawn markers in image regions that discriminate classes. It does not rely on optimization based on backpropagation, and we demonstrate its advantages on the binary classification of coconut-tree aerial images.
  arXiv  Detail & Related papers  (2020-08-08T15:50:23Z)
• I Am Going MAD: Maximum Discrepancy Competition for Comparing Classifiers Adaptively [135.7695909882746]
  We name the MAximum Discrepancy (MAD) competition. We adaptively sample a small test set from an arbitrarily large corpus of unlabeled images. Human labeling on the resulting model-dependent image sets reveals the relative performance of the competing classifiers.
  arXiv  Detail & Related papers  (2020-02-25T03:32:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.