Unravelling Small Sample Size Problems in the Deep Learning World

• URL: http://arxiv.org/abs/2008.03522v1
• Date: Sat, 8 Aug 2020 13:35:49 GMT
• Title: Unravelling Small Sample Size Problems in the Deep Learning World
• Authors: Rohit Keshari, Soumyadeep Ghosh, Saheb Chhabra, Mayank Vatsa, Richa Singh
• Abstract summary: We first present a review of deep learning algorithms for small sample size problems in which the algorithms are segregated according to the space in which they operate. Secondly, we present Dynamic Attention Pooling approach which focuses on extracting global information from the most discriminative sub-part of the feature map.
• Score: 69.82853912238173
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The growth and success of deep learning approaches can be attributed to two major factors: availability of hardware resources and availability of large number of training samples. For problems with large training databases, deep learning models have achieved superlative performances. However, there are a lot of \textit{small sample size or $S^3$} problems for which it is not feasible to collect large training databases. It has been observed that deep learning models do not generalize well on $S^3$ problems and specialized solutions are required. In this paper, we first present a review of deep learning algorithms for small sample size problems in which the algorithms are segregated according to the space in which they operate, i.e. input space, model space, and feature space. Secondly, we present Dynamic Attention Pooling approach which focuses on extracting global information from the most discriminative sub-part of the feature map. The performance of the proposed dynamic attention pooling is analyzed with state-of-the-art ResNet model on relatively small publicly available datasets such as SVHN, C10, C100, and TinyImageNet.

Related papers

• Towards Scalable and Versatile Weight Space Learning [51.78426981947659]
  This paper introduces the SANE approach to weight-space learning. Our method extends the idea of hyper-representations towards sequential processing of subsets of neural network weights.
  arXiv  Detail & Related papers  (2024-06-14T13:12:07Z)
• Scalable Learning of Item Response Theory Models [48.91265296134559]
  Item Response Theory (IRT) models aim to assess latent abilities of $n$ examinees along with latent difficulty characteristics of $m$ test items from categorical data. We leverage the similarity of these models to logistic regression, which can be approximated accurately using small weighted subsets called coresets.
  arXiv  Detail & Related papers  (2024-03-01T17:12:53Z)
• Interpretable ML for Imbalanced Data [22.355966235617014]
  Imbalanced data compounds the black-box nature of deep networks because the relationships between classes may be skewed and unclear. Existing methods that investigate imbalanced data complexity are geared toward binary classification, shallow learning models and low dimensional data. We propose a set of techniques that can be used by both deep learning model users to identify, visualize and understand class prototypes, sub-concepts and outlier instances.
  arXiv  Detail & Related papers  (2022-12-15T11:50:31Z)
• Learning the Space of Deep Models [12.785442275745973]
  We show how it is possible to use representation learning to learn a fixed-size, low-dimensional embedding space of trained deep models. We address image classification and neural representation of signals, showing how our embedding space can be learnt so as to capture the notions of performance and 3D shape.
  arXiv  Detail & Related papers  (2022-06-10T15:53:35Z)
• CHALLENGER: Training with Attribution Maps [63.736435657236505]
  We show that utilizing attribution maps for training neural networks can improve regularization of models and thus increase performance. In particular, we show that our generic domain-independent approach yields state-of-the-art results in vision, natural language processing and on time series tasks.
  arXiv  Detail & Related papers  (2022-05-30T13:34:46Z)
• CvS: Classification via Segmentation For Small Datasets [52.821178654631254]
  This paper presents CvS, a cost-effective classifier for small datasets that derives the classification labels from predicting the segmentation maps. We evaluate the effectiveness of our framework on diverse problems showing that CvS is able to achieve much higher classification results compared to previous methods when given only a handful of examples.
  arXiv  Detail & Related papers  (2021-10-29T18:41:15Z)
• Few-Cost Salient Object Detection with Adversarial-Paced Learning [95.0220555274653]
  This paper proposes to learn the effective salient object detection model based on the manual annotation on a few training images only. We name this task as the few-cost salient object detection and propose an adversarial-paced learning (APL)-based framework to facilitate the few-cost learning scenario.
  arXiv  Detail & Related papers  (2021-04-05T14:15:49Z)
• Spatial Classification With Limited Observations Based On Physics-Aware Structural Constraint [18.070762916388272]
  spatial classification with limited feature observations has been a challenging problem in machine learning. This paper extends our recent approach by allowing feature values of samples in each class to follow a multi-modal distribution. We propose learning algorithms for the extended model with multi-modal distribution.
  arXiv  Detail & Related papers  (2020-08-25T20:07:28Z)
• Reinforced active learning for image segmentation [34.096237671643145]
  We present a new active learning strategy for semantic segmentation based on deep reinforcement learning (RL) An agent learns a policy to select a subset of small informative image regions -- opposed to entire images -- to be labeled from a pool of unlabeled data. Our method proposes a new modification of the deep Q-network (DQN) formulation for active learning, adapting it to the large-scale nature of semantic segmentation problems.
  arXiv  Detail & Related papers  (2020-02-16T14:03:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.