SelectScale: Mining More Patterns from Images via Selective and Soft Dropout

• URL: http://arxiv.org/abs/2012.15766v1
• Date: Mon, 30 Nov 2020 12:15:08 GMT
• Title: SelectScale: Mining More Patterns from Images via Selective and Soft Dropout
• Authors: Zhengsu Chen, Jianwei Niu, Xuefeng Liu and Shaojie Tang
• Abstract summary: Convolutional neural networks (CNNs) have achieved remarkable success in image recognition. We propose SelectScale, which selects the important features in networks and adjusts them during training. Using SelectScale, we improve the performance of CNNs on CIFAR and ImageNet.
• Score: 35.066419181817594
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Convolutional neural networks (CNNs) have achieved remarkable success in image recognition. Although the internal patterns of the input images are effectively learned by the CNNs, these patterns only constitute a small proportion of useful patterns contained in the input images. This can be attributed to the fact that the CNNs will stop learning if the learned patterns are enough to make a correct classification. Network regularization methods like dropout and SpatialDropout can ease this problem. During training, they randomly drop the features. These dropout methods, in essence, change the patterns learned by the networks, and in turn, forces the networks to learn other patterns to make the correct classification. However, the above methods have an important drawback. Randomly dropping features is generally inefficient and can introduce unnecessary noise. To tackle this problem, we propose SelectScale. Instead of randomly dropping units, SelectScale selects the important features in networks and adjusts them during training. Using SelectScale, we improve the performance of CNNs on CIFAR and ImageNet.

Related papers

• Training Convolutional Neural Networks with the Forward-Forward algorithm [1.74440662023704]
  Forward Forward (FF) algorithm has up to now only been used in fully connected networks. We show how the FF paradigm can be extended to CNNs. Our FF-trained CNN, featuring a novel spatially-extended labeling technique, achieves a classification accuracy of 99.16% on the MNIST hand-written digits dataset.
  arXiv  Detail & Related papers  (2023-12-22T18:56:35Z)
• Random Padding Data Augmentation [23.70951896315126]
  convolutional neural network (CNN) learns the same object in different positions in images. The usefulness of the features' spatial information in CNNs has not been well investigated. We introduce Random Padding, a new type of padding method for training CNNs.
  arXiv  Detail & Related papers  (2023-02-17T04:15:33Z)
• Background Invariant Classification on Infrared Imagery by Data Efficient Training and Reducing Bias in CNNs [1.2891210250935146]
  convolutional neural networks can classify objects in images very accurately. It is well known that the attention of the network may not always be on the semantically important regions of the scene. We propose a new two-step training procedure called textitsplit training to reduce this bias in CNNs on both Infrared imagery and RGB data.
  arXiv  Detail & Related papers  (2022-01-22T23:29:42Z)
• Learning to ignore: rethinking attention in CNNs [87.01305532842878]
  We propose to reformulate the attention mechanism in CNNs to learn to ignore instead of learning to attend. Specifically, we propose to explicitly learn irrelevant information in the scene and suppress it in the produced representation.
  arXiv  Detail & Related papers  (2021-11-10T13:47:37Z)
• The Mind's Eye: Visualizing Class-Agnostic Features of CNNs [92.39082696657874]
  We propose an approach to visually interpret CNN features given a set of images by creating corresponding images that depict the most informative features of a specific layer. Our method uses a dual-objective activation and distance loss, without requiring a generator network nor modifications to the original model.
  arXiv  Detail & Related papers  (2021-01-29T07:46:39Z)
• AutoDropout: Learning Dropout Patterns to Regularize Deep Networks [82.28118615561912]
  Dropout or weight decay methods do not leverage the structures of the network's inputs and hidden states. We show that this method works well for both image recognition on CIFAR-10 and ImageNet, as well as language modeling on Penn Treebank and WikiText-2. The learned dropout patterns also transfers to different tasks and datasets, such as from language model on Penn Treebank to Engligh-French translation on WMT 2014.
  arXiv  Detail & Related papers  (2021-01-05T19:54:22Z)
• Shape-Texture Debiased Neural Network Training [50.6178024087048]
  Convolutional Neural Networks are often biased towards either texture or shape, depending on the training dataset. We develop an algorithm for shape-texture debiased learning. Experiments show that our method successfully improves model performance on several image recognition benchmarks.
  arXiv  Detail & Related papers  (2020-10-12T19:16:12Z)
• Adaptive Label Smoothing [1.3198689566654107]
  We present a novel approach to classification that combines the ideas of objectness and label smoothing during training. We show extensive results using ImageNet to demonstrate that CNNs trained using adaptive label smoothing are much less likely to be overconfident in their predictions.
  arXiv  Detail & Related papers  (2020-09-14T13:37:30Z)
• Learning CNN filters from user-drawn image markers for coconut-tree image classification [78.42152902652215]
  We present a method that needs a minimal set of user-selected images to train the CNN's feature extractor. The method learns the filters of each convolutional layer from user-drawn markers in image regions that discriminate classes. It does not rely on optimization based on backpropagation, and we demonstrate its advantages on the binary classification of coconut-tree aerial images.
  arXiv  Detail & Related papers  (2020-08-08T15:50:23Z)
• Learning to Learn Parameterized Classification Networks for Scalable Input Images [76.44375136492827]
  Convolutional Neural Networks (CNNs) do not have a predictable recognition behavior with respect to the input resolution change. We employ meta learners to generate convolutional weights of main networks for various input scales. We further utilize knowledge distillation on the fly over model predictions based on different input resolutions.
  arXiv  Detail & Related papers  (2020-07-13T04:27:25Z)
• Retrain or not retrain? -- efficient pruning methods of deep CNN networks [0.30458514384586394]
  Convolutional neural networks (CNN) play a major role in image processing tasks like image classification, object detection, semantic segmentation. Very often CNN networks have from several to hundred stacked layers with several megabytes of weights. One of the possible methods to reduce complexity and memory footprint is pruning.
  arXiv  Detail & Related papers  (2020-02-12T23:24:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.