Related papers: Long-tail learning via logit adjustment

Long-tail learning via logit adjustment

URL: http://arxiv.org/abs/2007.07314v2
Date: Fri, 9 Jul 2021 21:23:25 GMT
Title: Long-tail learning via logit adjustment
Authors: Aditya Krishna Menon and Sadeep Jayasumana and Ankit Singh Rawat and Himanshu Jain and Andreas Veit and Sanjiv Kumar
Abstract summary: Real-world classification problems typically exhibit an imbalanced or long-tailed label distribution. This poses a challenge for generalisation on such labels, and also makes na"ive learning biased towards dominant labels. We present two simple modifications of standard softmax cross-entropy training to cope with these challenges.
Score: 67.47668112425225
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Abstract: Real-world classification problems typically exhibit an imbalanced or long-tailed label distribution, wherein many labels are associated with only a few samples. This poses a challenge for generalisation on such labels, and also makes na\"ive learning biased towards dominant labels. In this paper, we present two simple modifications of standard softmax cross-entropy training to cope with these challenges. Our techniques revisit the classic idea of logit adjustment based on the label frequencies, either applied post-hoc to a trained model, or enforced in the loss during training. Such adjustment encourages a large relative margin between logits of rare versus dominant labels. These techniques unify and generalise several recent proposals in the literature, while possessing firmer statistical grounding and empirical performance.

Related papers

Exploring Vacant Classes in Label-Skewed Federated Learning [113.65301899666645]
Label skews, characterized by disparities in local label distribution across clients, pose a significant challenge in federated learning. This paper introduces FedVLS, a novel approach to label-skewed federated learning that integrates vacant-class distillation and logit suppression simultaneously.
arXiv Detail & Related papers (2024-01-04T16:06:31Z)
Class-Distribution-Aware Pseudo Labeling for Semi-Supervised Multi-Label Learning [97.88458953075205]
Pseudo-labeling has emerged as a popular and effective approach for utilizing unlabeled data. This paper proposes a novel solution called Class-Aware Pseudo-Labeling (CAP) that performs pseudo-labeling in a class-aware manner.
arXiv Detail & Related papers (2023-05-04T12:52:18Z)
Towards Calibrated Model for Long-Tailed Visual Recognition from Prior Perspective [17.733087434470907]
Real-world data universally confronts a severe class-imbalance problem and exhibits a long-tailed distribution. We propose two novel methods from the prior perspective to alleviate this dilemma. First, we deduce a balance-oriented data augmentation named Uniform Mixup (UniMix) to promote mixup in long-tailed scenarios. Second, motivated by the Bayesian theory, we figure out the Bayes Bias (Bayias) to compensate it as a modification on standard cross-entropy loss.
arXiv Detail & Related papers (2021-11-06T12:53:34Z)
An Empirical Investigation of Learning from Biased Toxicity Labels [15.822714574671412]
We study how different training strategies can leverage a small dataset of human-annotated labels and a large but noisy dataset of synthetically generated labels. We evaluate the accuracy and fairness properties of these approaches, and trade-offs between the two.
arXiv Detail & Related papers (2021-10-04T17:19:57Z)
Robust Long-Tailed Learning under Label Noise [50.00837134041317]
This work investigates the label noise problem under long-tailed label distribution. We propose a robust framework,algo, that realizes noise detection for long-tailed learning. Our framework can naturally leverage semi-supervised learning algorithms to further improve the generalisation.
arXiv Detail & Related papers (2021-08-26T03:45:00Z)
Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces [64.23172847182109]
We show that different negative sampling schemes implicitly trade-off performance on dominant versus rare labels. We provide a unified means to explicitly tackle both sampling bias, arising from working with a subset of all labels, and labeling bias, which is inherent to the data due to label imbalance.
arXiv Detail & Related papers (2021-05-12T15:40:13Z)
Harmless label noise and informative soft-labels in supervised classification [1.6752182911522517]
Manual labelling of training examples is common practice in supervised learning. When the labelling task is of non-trivial difficulty, the supplied labels may not be equal to the ground-truth labels, and label noise is introduced into the training dataset. In particular, when classification difficulty is the only source of label errors, multiple sets of noisy labels can supply more information for the estimation of a classification rule.
arXiv Detail & Related papers (2021-04-07T02:56:11Z)
Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets [98.74153364118898]
We present a new loss function called Distribution-Balanced Loss for the multi-label recognition problems that exhibit long-tailed class distributions. The Distribution-Balanced Loss tackles these issues through two key modifications to the standard binary cross-entropy loss. Experiments on both Pascal VOC and COCO show that the models trained with this new loss function achieve significant performance gains.
arXiv Detail & Related papers (2020-07-19T11:50:10Z)
Rethinking the Value of Labels for Improving Class-Imbalanced Learning [20.953282288425118]
Class-imbalanced learning can significantly benefit in both semi-supervised and self-supervised manners. We argue that imbalanced labels are not useful always. Our findings highlight the need to rethink the usage of imbalanced labels in realistic long-tailed tasks.
arXiv Detail & Related papers (2020-06-13T01:35:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.