Label-Aware Distribution Calibration for Long-tailed Classification
- URL: http://arxiv.org/abs/2111.04901v1
- Date: Tue, 9 Nov 2021 01:38:35 GMT
- Title: Label-Aware Distribution Calibration for Long-tailed Classification
- Authors: Chaozheng Wang, Shuzheng Gao, Cuiyun Gao, Pengyun Wang, Wenjie Pei,
Lujia Pan, Zenglin Xu
- Abstract summary: We propose a label-Aware Distribution LADC approach to calibrate the distribution of tail classes.
Experiments on both image and text long-tailed datasets demonstrate that LADC significantly outperforms existing methods.
- Score: 25.588323749920324
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Real-world data usually present long-tailed distributions. Training on
imbalanced data tends to render neural networks perform well on head classes
while much worse on tail classes. The severe sparseness of training instances
for the tail classes is the main challenge, which results in biased
distribution estimation during training. Plenty of efforts have been devoted to
ameliorating the challenge, including data re-sampling and synthesizing new
training instances for tail classes. However, no prior research has exploited
the transferable knowledge from head classes to tail classes for calibrating
the distribution of tail classes. In this paper, we suppose that tail classes
can be enriched by similar head classes and propose a novel distribution
calibration approach named as label-Aware Distribution Calibration LADC. LADC
transfers the statistics from relevant head classes to infer the distribution
of tail classes. Sampling from calibrated distribution further facilitates
re-balancing the classifier. Experiments on both image and text long-tailed
datasets demonstrate that LADC significantly outperforms existing methods.The
visualization also shows that LADC provides a more accurate distribution
estimation.
Related papers
- Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition [50.61991746981703]
Current state-of-the-art LTSSL approaches rely on high-quality pseudo-labels for large-scale unlabeled data.
This paper introduces a novel probabilistic framework that unifies various recent proposals in long-tail learning.
We introduce a continuous contrastive learning method, CCL, extending our framework to unlabeled data using reliable and smoothed pseudo-labels.
arXiv Detail & Related papers (2024-10-08T15:06:10Z) - Dual Compensation Residual Networks for Class Imbalanced Learning [98.35401757647749]
We propose Dual Compensation Residual Networks to better fit both tail and head classes.
An important factor causing overfitting is that there is severe feature drift between training and test data on tail classes.
We also propose a Residual Balanced Multi-Proxies classifier to alleviate the under-fitting issue.
arXiv Detail & Related papers (2023-08-25T04:06:30Z) - Adjusting Logit in Gaussian Form for Long-Tailed Visual Recognition [37.62659619941791]
We study the problem of long-tailed visual recognition from the perspective of feature level.
Two novel logit adjustment methods are proposed to improve model performance at a modest computational overhead.
Experiments conducted on benchmark datasets demonstrate the superior performance of the proposed method over the state-of-the-art ones.
arXiv Detail & Related papers (2023-05-18T02:06:06Z) - Transfer Knowledge from Head to Tail: Uncertainty Calibration under
Long-tailed Distribution [24.734851889816206]
Current calibration techniques treat different classes equally and implicitly assume that the distribution of training data is balanced.
We propose a novel knowledge-transferring-based calibration method by estimating the importance weights for samples of tail classes.
arXiv Detail & Related papers (2023-04-13T13:48:18Z) - When Noisy Labels Meet Long Tail Dilemmas: A Representation Calibration
Method [40.25499257944916]
Real-world datasets are both noisily labeled and class-imbalanced.
We propose a representation calibration method RCAL.
We derive theoretical results to discuss the effectiveness of our representation calibration.
arXiv Detail & Related papers (2022-11-20T11:36:48Z) - Learnable Distribution Calibration for Few-Shot Class-Incremental
Learning [122.2241120474278]
Few-shot class-incremental learning (FSCIL) faces challenges of memorizing old class distributions and estimating new class distributions given few training samples.
We propose a learnable distribution calibration (LDC) approach, with the aim to systematically solve these two challenges using a unified framework.
arXiv Detail & Related papers (2022-10-01T09:40:26Z) - Learning Muti-expert Distribution Calibration for Long-tailed Video
Classification [88.12433458277168]
We propose an end-to-end multi-experts distribution calibration method based on two-level distribution information.
By modeling this two-level distribution information, the model can consider the head classes and the tail classes.
Our method achieves state-of-the-art performance on the long-tailed video classification task.
arXiv Detail & Related papers (2022-05-22T09:52:34Z) - Calibrating Class Activation Maps for Long-Tailed Visual Recognition [60.77124328049557]
We present two effective modifications of CNNs to improve network learning from long-tailed distribution.
First, we present a Class Activation Map (CAMC) module to improve the learning and prediction of network classifiers.
Second, we investigate the use of normalized classifiers for representation learning in long-tailed problems.
arXiv Detail & Related papers (2021-08-29T05:45:03Z) - Distributional Robustness Loss for Long-tail Learning [20.800627115140465]
Real-world data is often unbalanced and long-tailed, but deep models struggle to recognize rare classes in the presence of frequent classes.
We show that the feature extractor part of deep networks suffers greatly from this bias.
We propose a new loss based on robustness theory, which encourages the model to learn high-quality representations for both head and tail classes.
arXiv Detail & Related papers (2021-04-07T11:34:04Z) - Long-tailed Recognition by Routing Diverse Distribution-Aware Experts [64.71102030006422]
We propose a new long-tailed classifier called RoutIng Diverse Experts (RIDE)
It reduces the model variance with multiple experts, reduces the model bias with a distribution-aware diversity loss, reduces the computational cost with a dynamic expert routing module.
RIDE outperforms the state-of-the-art by 5% to 7% on CIFAR100-LT, ImageNet-LT and iNaturalist 2018 benchmarks.
arXiv Detail & Related papers (2020-10-05T06:53:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.