Simplifying Neural Network Training Under Class Imbalance

• URL: http://arxiv.org/abs/2312.02517v1
• Date: Tue, 5 Dec 2023 05:52:44 GMT
• Title: Simplifying Neural Network Training Under Class Imbalance
• Authors: Ravid Shwartz-Ziv and Micah Goldblum and Yucen Lily Li and C. Bayan Bruss and Andrew Gordon Wilson
• Abstract summary: Real-world datasets are often highly class-imbalanced, which can adversely impact the performance of deep learning models. The majority of research on training neural networks under class imbalance has focused on specialized loss functions, sampling techniques, or two-stage training procedures. We demonstrate that simply tuning existing components of standard deep learning pipelines, such as the batch size, data augmentation, and label smoothing, can achieve state-of-the-art performance without any such specialized class imbalance methods.
• Score: 77.39968702907817
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world datasets are often highly class-imbalanced, which can adversely impact the performance of deep learning models. The majority of research on training neural networks under class imbalance has focused on specialized loss functions, sampling techniques, or two-stage training procedures. Notably, we demonstrate that simply tuning existing components of standard deep learning pipelines, such as the batch size, data augmentation, optimizer, and label smoothing, can achieve state-of-the-art performance without any such specialized class imbalance methods. We also provide key prescriptions and considerations for training under class imbalance, and an understanding of why imbalance methods succeed or fail.

Related papers

• Efficient Augmentation for Imbalanced Deep Learning [8.38844520504124]
  We study a convolutional neural network's internal representation of imbalanced image data. We measure the generalization gap between a model's feature embeddings in the training and test sets, showing that the gap is wider for minority classes. This insight enables us to design an efficient three-phase CNN training framework for imbalanced data.
  arXiv  Detail & Related papers  (2022-07-13T09:43:17Z)
• Phased Progressive Learning with Coupling-Regulation-Imbalance Loss for Imbalanced Classification [11.673344551762822]
  Deep neural networks generally perform poorly with datasets that suffer from quantity imbalance and classification difficulty imbalance between different classes. A phased progressive learning schedule was proposed for smoothly transferring the training emphasis from representation learning to upper classifier training. Our code will be open source soon.
  arXiv  Detail & Related papers  (2022-05-24T14:46:39Z)
• Deep Reinforcement Learning for Multi-class Imbalanced Training [64.9100301614621]
  We introduce an imbalanced classification framework, based on reinforcement learning, for training extremely imbalanced data sets. We formulate a custom reward function and episode-training procedure, specifically with the added capability of handling multi-class imbalanced training. Using real-world clinical case studies, we demonstrate that our proposed framework outperforms current state-of-the-art imbalanced learning methods.
  arXiv  Detail & Related papers  (2022-05-24T13:39:59Z)
• CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning [55.733193075728096]
  Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance. Sample re-weighting methods are popularly used to alleviate this data bias issue. We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
  arXiv  Detail & Related papers  (2022-02-11T13:49:51Z)
• Prototypical Classifier for Robust Class-Imbalanced Learning [64.96088324684683]
  We propose textitPrototypical, which does not require fitting additional parameters given the embedding network. Prototypical produces balanced and comparable predictions for all classes even though the training set is class-imbalanced. We test our method on CIFAR-10LT, CIFAR-100LT and Webvision datasets, observing that Prototypical obtains substaintial improvements compared with state of the arts.
  arXiv  Detail & Related papers  (2021-10-22T01:55:01Z)
• Towards Balanced Learning for Instance Recognition [149.76724446376977]
  We propose Libra R-CNN, a framework towards balanced learning for instance recognition. It integrates IoU-balanced sampling, balanced feature pyramid, and objective re-weighting, respectively for reducing the imbalance at sample, feature, and objective level.
  arXiv  Detail & Related papers  (2021-08-23T13:40:45Z)
• MetaBalance: High-Performance Neural Networks for Class-Imbalanced Data [42.81296448544681]
  Class-imbalanced data, in which some classes contain far more samples than others, is ubiquitous in real-world applications. Standard techniques for handling class-imbalance usually work by training on a re-weighted loss or on re-balanced data. We evaluate our method, MetaBalance, on image classification, credit-card fraud detection, loan default prediction, and facial recognition tasks with severely imbalanced data.
  arXiv  Detail & Related papers  (2021-06-17T16:42:50Z)
• Mitigating Dataset Imbalance via Joint Generation and Classification [17.57577266707809]
  Supervised deep learning methods are enjoying enormous success in many practical applications of computer vision. The marked performance degradation to biases and imbalanced data questions the reliability of these methods. We introduce a joint dataset repairment strategy by combining a neural network classifier with Generative Adversarial Networks (GAN) We show that the combined training helps to improve the robustness of both the classifier and the GAN against severe class imbalance.
  arXiv  Detail & Related papers  (2020-08-12T18:40:38Z)
• Long-Tailed Recognition Using Class-Balanced Experts [128.73438243408393]
  We propose an ensemble of class-balanced experts that combines the strength of diverse classifiers. Our ensemble of class-balanced experts reaches results close to state-of-the-art and an extended ensemble establishes a new state-of-the-art on two benchmarks for long-tailed recognition.
  arXiv  Detail & Related papers  (2020-04-07T20:57:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.