A Data-Centric Approach for Training Deep Neural Networks with Less Data

• URL: http://arxiv.org/abs/2110.03613v1
• Date: Thu, 7 Oct 2021 16:41:52 GMT
• Title: A Data-Centric Approach for Training Deep Neural Networks with Less Data
• Authors: Mohammad Motamedi, Nikolay Sakharnykh, Tim Kaldewey
• Abstract summary: This paper summarizes our winning submission to the "Data-Centric AI" competition. We discuss some of the challenges that arise while training with a small dataset. We propose a GAN-based solution for synthesizing new data points.
• Score: 1.9014535120129343
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While the availability of large datasets is perceived to be a key requirement for training deep neural networks, it is possible to train such models with relatively little data. However, compensating for the absence of large datasets demands a series of actions to enhance the quality of the existing samples and to generate new ones. This paper summarizes our winning submission to the "Data-Centric AI" competition. We discuss some of the challenges that arise while training with a small dataset, offer a principled approach for systematic data quality enhancement, and propose a GAN-based solution for synthesizing new data points. Our evaluations indicate that the dataset generated by the proposed pipeline offers 5% accuracy improvement while being significantly smaller than the baseline.

Related papers

• A CLIP-Powered Framework for Robust and Generalizable Data Selection [51.46695086779598]
  Real-world datasets often contain redundant and noisy data, imposing a negative impact on training efficiency and model performance. Data selection has shown promise in identifying the most representative samples from the entire dataset. We propose a novel CLIP-powered data selection framework that leverages multimodal information for more robust and generalizable sample selection.
  arXiv  Detail & Related papers  (2024-10-15T03:00:58Z)
• How Much Data are Enough? Investigating Dataset Requirements for Patch-Based Brain MRI Segmentation Tasks [74.21484375019334]
  Training deep neural networks reliably requires access to large-scale datasets. To mitigate both the time and financial costs associated with model development, a clear understanding of the amount of data required to train a satisfactory model is crucial. This paper proposes a strategic framework for estimating the amount of annotated data required to train patch-based segmentation networks.
  arXiv  Detail & Related papers  (2024-04-04T13:55:06Z)
• Data Filtering Networks [67.827994353269]
  We study the problem of learning a data filtering network (DFN) for this second step of filtering a large uncurated dataset. Our key finding is that the quality of a network for filtering is distinct from its performance on downstream tasks. Based on our insights, we construct new data filtering networks that induce state-of-the-art image-text datasets.
  arXiv  Detail & Related papers  (2023-09-29T17:37:29Z)
• Iterative self-transfer learning: A general methodology for response time-history prediction based on small dataset [0.0]
  An iterative self-transfer learningmethod for training neural networks based on small datasets is proposed in this study. The results show that the proposed method can improve the model performance by near an order of magnitude on small datasets.
  arXiv  Detail & Related papers  (2023-06-14T18:48:04Z)
• Dataset Distillation: A Comprehensive Review [76.26276286545284]
  dataset distillation (DD) aims to derive a much smaller dataset containing synthetic samples, based on which the trained models yield performance comparable with those trained on the original dataset. This paper gives a comprehensive review and summary of recent advances in DD and its application.
  arXiv  Detail & Related papers  (2023-01-17T17:03:28Z)
• A Proposal to Study "Is High Quality Data All We Need?" [8.122270502556374]
  We propose an empirical study that examines how to select a subset of and/or create high quality benchmark data. We seek to answer if big datasets are truly needed to learn a task, and whether a smaller subset of high quality data can replace big datasets.
  arXiv  Detail & Related papers  (2022-03-12T10:50:13Z)
• A Deep-Learning Intelligent System Incorporating Data Augmentation for Short-Term Voltage Stability Assessment of Power Systems [9.299576471941753]
  This paper proposes a novel deep-learning intelligent system incorporating data augmentation for STVSA of power systems. Due to the unavailability of reliable quantitative criteria to judge the stability status for a specific power system, semi-supervised cluster learning is leveraged to obtain labeled samples. conditional least squares generative adversarial networks (LSGAN)-based data augmentation is introduced to expand the original dataset.
  arXiv  Detail & Related papers  (2021-12-05T11:40:54Z)
• The Imaginative Generative Adversarial Network: Automatic Data Augmentation for Dynamic Skeleton-Based Hand Gesture and Human Action Recognition [27.795763107984286]
  We present a novel automatic data augmentation model, which approximates the distribution of the input data and samples new data from this distribution. Our results show that the augmentation strategy is fast to train and can improve classification accuracy for both neural networks and state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-05-27T11:07:09Z)
• Dataset Meta-Learning from Kernel Ridge-Regression [18.253682891579402]
  Kernel Inducing Points (KIP) can compress datasets by one or two orders of magnitude. KIP-learned datasets are transferable to the training of finite-width neural networks even beyond the lazy-training regime.
  arXiv  Detail & Related papers  (2020-10-30T18:54:04Z)
• On Robustness and Transferability of Convolutional Neural Networks [147.71743081671508]
  Modern deep convolutional networks (CNNs) are often criticized for not generalizing under distributional shifts. We study the interplay between out-of-distribution and transfer performance of modern image classification CNNs for the first time. We find that increasing both the training set and model sizes significantly improve the distributional shift robustness.
  arXiv  Detail & Related papers  (2020-07-16T18:39:04Z)
• DeGAN : Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier [58.979104709647295]
  We bridge the gap between the abundance of available data and lack of relevant data, for the future learning tasks of a trained network. We use the available data, that may be an imbalanced subset of the original training dataset, or a related domain dataset, to retrieve representative samples. We demonstrate that data from a related domain can be leveraged to achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2019-12-27T02:05:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.