Class-Wise Difficulty-Balanced Loss for Solving Class-Imbalance
- URL: http://arxiv.org/abs/2010.01824v1
- Date: Mon, 5 Oct 2020 07:19:19 GMT
- Title: Class-Wise Difficulty-Balanced Loss for Solving Class-Imbalance
- Authors: Saptarshi Sinha, Hiroki Ohashi and Katsuyuki Nakamura
- Abstract summary: We propose a novel loss function named Class-wise Difficulty-Balanced loss.
It dynamically distributes weights to each sample according to the difficulty of the class that the sample belongs to.
The results show that CDB loss consistently outperforms the recently proposed loss functions on class-imbalanced datasets.
- Score: 6.875312133832079
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Class-imbalance is one of the major challenges in real world datasets, where
a few classes (called majority classes) constitute much more data samples than
the rest (called minority classes). Learning deep neural networks using such
datasets leads to performances that are typically biased towards the majority
classes. Most of the prior works try to solve class-imbalance by assigning more
weights to the minority classes in various manners (e.g., data re-sampling,
cost-sensitive learning). However, we argue that the number of available
training data may not be always a good clue to determine the weighting strategy
because some of the minority classes might be sufficiently represented even by
a small number of training data. Overweighting samples of such classes can lead
to drop in the model's overall performance. We claim that the 'difficulty' of a
class as perceived by the model is more important to determine the weighting.
In this light, we propose a novel loss function named Class-wise
Difficulty-Balanced loss, or CDB loss, which dynamically distributes weights to
each sample according to the difficulty of the class that the sample belongs
to. Note that the assigned weights dynamically change as the 'difficulty' for
the model may change with the learning progress. Extensive experiments are
conducted on both image (artificially induced class-imbalanced MNIST,
long-tailed CIFAR and ImageNet-LT) and video (EGTEA) datasets. The results show
that CDB loss consistently outperforms the recently proposed loss functions on
class-imbalanced datasets irrespective of the data type (i.e., video or image).
Related papers
- A Unified Generalization Analysis of Re-Weighting and Logit-Adjustment
for Imbalanced Learning [129.63326990812234]
We propose a technique named data-dependent contraction to capture how modified losses handle different classes.
On top of this technique, a fine-grained generalization bound is established for imbalanced learning, which helps reveal the mystery of re-weighting and logit-adjustment.
arXiv Detail & Related papers (2023-10-07T09:15:08Z) - Learning to Re-weight Examples with Optimal Transport for Imbalanced
Classification [74.62203971625173]
Imbalanced data pose challenges for deep learning based classification models.
One of the most widely-used approaches for tackling imbalanced data is re-weighting.
We propose a novel re-weighting method based on optimal transport (OT) from a distributional point of view.
arXiv Detail & Related papers (2022-08-05T01:23:54Z) - Class-Difficulty Based Methods for Long-Tailed Visual Recognition [6.875312133832079]
Long-tailed datasets are frequently encountered in real-world use cases where few classes or categories have higher number of data samples compared to the other classes.
We propose a novel approach to dynamically measure the instantaneous difficulty of each class during the training phase of the model.
We also use the difficulty measures of each class to design a novel weighted loss technique called class-wise difficulty based weighted' and a novel data sampling technique called class-wise difficulty based sampling'
arXiv Detail & Related papers (2022-07-29T06:33:22Z) - Throwing Away Data Improves Worst-Class Error in Imbalanced
Classification [36.91428748713018]
Class imbalances pervade classification problems, yet their treatment differs in theory and practice.
We take on the challenge of developing learning theory able to describe the worst-class error of classifiers over linearly-separable data.
arXiv Detail & Related papers (2022-05-23T23:43:18Z) - Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for
Imbalanced Learning [97.81549071978789]
We propose Attraction-Repulsion-Balanced Loss (ARB-Loss) to balance the different components of the gradients.
We perform experiments on the large-scale classification and segmentation datasets and our ARB-Loss can achieve state-of-the-art performance.
arXiv Detail & Related papers (2022-04-19T08:23:23Z) - Imbalanced Classification via Explicit Gradient Learning From Augmented
Data [0.0]
We propose a novel deep meta-learning technique to augment a given imbalanced dataset with new minority instances.
The advantage of the proposed method is demonstrated on synthetic and real-world datasets with various imbalance ratios.
arXiv Detail & Related papers (2022-02-21T22:16:50Z) - CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep
Learning [55.733193075728096]
Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance.
Sample re-weighting methods are popularly used to alleviate this data bias issue.
We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
arXiv Detail & Related papers (2022-02-11T13:49:51Z) - Self-Damaging Contrastive Learning [92.34124578823977]
Unlabeled data in reality is commonly imbalanced and shows a long-tail distribution.
This paper proposes a principled framework called Self-Damaging Contrastive Learning to automatically balance the representation learning without knowing the classes.
Our experiments show that SDCLR significantly improves not only overall accuracies but also balancedness.
arXiv Detail & Related papers (2021-06-06T00:04:49Z) - Distributional Robustness Loss for Long-tail Learning [20.800627115140465]
Real-world data is often unbalanced and long-tailed, but deep models struggle to recognize rare classes in the presence of frequent classes.
We show that the feature extractor part of deep networks suffers greatly from this bias.
We propose a new loss based on robustness theory, which encourages the model to learn high-quality representations for both head and tail classes.
arXiv Detail & Related papers (2021-04-07T11:34:04Z) - M2m: Imbalanced Classification via Major-to-minor Translation [79.09018382489506]
In most real-world scenarios, labeled training datasets are highly class-imbalanced, where deep neural networks suffer from generalizing to a balanced testing criterion.
In this paper, we explore a novel yet simple way to alleviate this issue by augmenting less-frequent classes via translating samples from more-frequent classes.
Our experimental results on a variety of class-imbalanced datasets show that the proposed method improves the generalization on minority classes significantly compared to other existing re-sampling or re-weighting methods.
arXiv Detail & Related papers (2020-04-01T13:21:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.