MSE Loss with Outlying Label for Imbalanced Classification
- URL: http://arxiv.org/abs/2107.02393v1
- Date: Tue, 6 Jul 2021 05:17:00 GMT
- Title: MSE Loss with Outlying Label for Imbalanced Classification
- Authors: Sota Kato, Kazuhiro Hotta
- Abstract summary: We propose mean squared error (MSE) loss with outlying label for class imbalanced classification.
MSE loss is possible to equalize the number of back propagation for all classes and to learn the feature space considering the relationships between classes as metric learning.
It is possible to create the feature space for separating high-difficulty classes and low-difficulty classes.
- Score: 10.305130700118399
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this paper, we propose mean squared error (MSE) loss with outlying label
for class imbalanced classification. Cross entropy (CE) loss, which is widely
used for image recognition, is learned so that the probability value of true
class is closer to one by back propagation. However, for imbalanced datasets,
the learning is insufficient for the classes with a small number of samples.
Therefore, we propose a novel classification method using the MSE loss that can
be learned the relationships of all classes no matter which image is input.
Unlike CE loss, MSE loss is possible to equalize the number of back propagation
for all classes and to learn the feature space considering the relationships
between classes as metric learning. Furthermore, instead of the usual one-hot
teacher label, we use a novel teacher label that takes the number of class
samples into account. This induces the outlying label which depends on the
number of samples in each class, and the class with a small number of samples
has outlying margin in a feature space. It is possible to create the feature
space for separating high-difficulty classes and low-difficulty classes. By the
experiments on imbalanced classification and semantic segmentation, we
confirmed that the proposed method was much improved in comparison with
standard CE loss and conventional methods, even though only the loss and
teacher labels were changed.
Related papers
- Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning [59.44422468242455]
We propose a novel method dubbed ShrinkMatch to learn uncertain samples.
For each uncertain sample, it adaptively seeks a shrunk class space, which merely contains the original top-1 class.
We then impose a consistency regularization between a pair of strongly and weakly augmented samples in the shrunk space to strive for discriminative representations.
arXiv Detail & Related papers (2023-08-13T14:05:24Z) - PercentMatch: Percentile-based Dynamic Thresholding for Multi-Label
Semi-Supervised Classification [64.39761523935613]
We propose a percentile-based threshold adjusting scheme to dynamically alter the score thresholds of positive and negative pseudo-labels for each class during the training.
We achieve strong performance on Pascal VOC2007 and MS-COCO datasets when compared to recent SSL methods.
arXiv Detail & Related papers (2022-08-30T01:27:48Z) - Learning from Label Proportions by Learning with Label Noise [30.7933303912474]
Learning from label proportions (LLP) is a weakly supervised classification problem where data points are grouped into bags.
We provide a theoretically grounded approach to LLP based on a reduction to learning with label noise.
Our approach demonstrates improved empirical performance in deep learning scenarios across multiple datasets and architectures.
arXiv Detail & Related papers (2022-03-04T18:52:21Z) - Shift Happens: Adjusting Classifiers [2.8682942808330703]
Minimizing expected loss measured by a proper scoring rule, such as Brier score or log-loss (cross-entropy), is a common objective while training a probabilistic classifier.
We propose methods that transform all predictions to (re)equalize the average prediction and the class distribution.
We demonstrate experimentally that, when in practice the class distribution is known only approximately, there is often still a reduction in loss depending on the amount of shift and the precision to which the class distribution is known.
arXiv Detail & Related papers (2021-11-03T21:27:27Z) - sigmoidF1: A Smooth F1 Score Surrogate Loss for Multilabel
Classification [42.37189502220329]
We propose a loss function, sigmoidF1, to account for the complexity of multilabel classification evaluation.
We show that sigmoidF1 outperforms other loss functions on four datasets and several metrics.
arXiv Detail & Related papers (2021-08-24T08:11:33Z) - Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced
Semi-Supervised Learning [80.05441565830726]
This paper addresses imbalanced semi-supervised learning, where heavily biased pseudo-labels can harm the model performance.
We propose a general pseudo-labeling framework to address the bias motivated by this observation.
We term the novel pseudo-labeling framework for imbalanced SSL as Distribution-Aware Semantics-Oriented (DASO) Pseudo-label.
arXiv Detail & Related papers (2021-06-10T11:58:25Z) - PLM: Partial Label Masking for Imbalanced Multi-label Classification [59.68444804243782]
Neural networks trained on real-world datasets with long-tailed label distributions are biased towards frequent classes and perform poorly on infrequent classes.
We propose a method, Partial Label Masking (PLM), which utilizes this ratio during training.
Our method achieves strong performance when compared to existing methods on both multi-label (MultiMNIST and MSCOCO) and single-label (imbalanced CIFAR-10 and CIFAR-100) image classification datasets.
arXiv Detail & Related papers (2021-05-22T18:07:56Z) - Exploiting Class Similarity for Machine Learning with Confidence Labels
and Projective Loss Functions [0.0]
Class labels are relatable to each other, with certain class labels being more similar to each other than others.
Current labeling techniques fail to explicitly capture such similarity information.
We use our approach to train neural networks with noisy labels, as we believe noisy labels are partly a result of confusability arising from class similarity.
arXiv Detail & Related papers (2021-03-25T04:49:44Z) - Rethinking preventing class-collapsing in metric learning with
margin-based losses [81.22825616879936]
Metric learning seeks embeddings where visually similar instances are close and dissimilar instances are apart.
margin-based losses tend to project all samples of a class onto a single point in the embedding space.
We propose a simple modification to the embedding losses such that each sample selects its nearest same-class counterpart in a batch.
arXiv Detail & Related papers (2020-06-09T09:59:25Z) - Negative Margin Matters: Understanding Margin in Few-shot Classification [72.85978953262004]
This paper introduces a negative margin loss to metric learning based few-shot learning methods.
The negative margin loss significantly outperforms regular softmax loss, and state-of-the-art accuracy on three standard few-shot classification benchmarks.
arXiv Detail & Related papers (2020-03-26T17:59:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.