Targeted Supervised Contrastive Learning for Long-Tailed Recognition
- URL: http://arxiv.org/abs/2111.13998v1
- Date: Sat, 27 Nov 2021 22:40:10 GMT
- Title: Targeted Supervised Contrastive Learning for Long-Tailed Recognition
- Authors: Tianhong Li, Peng Cao, Yuan Yuan, Lijie Fan, Yuzhe Yang, Rogerio
Feris, Piotr Indyk, Dina Katabi
- Abstract summary: Real-world data often exhibits long tail distributions with heavy class imbalance.
We show that while supervised contrastive learning can help improve performance, past baselines suffer from poor uniformity brought in by imbalanced data distribution.
We propose targeted supervised contrastive learning (TSC), which improves the uniformity of the feature distribution on the hypersphere.
- Score: 50.24044608432207
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Real-world data often exhibits long tail distributions with heavy class
imbalance, where the majority classes can dominate the training process and
alter the decision boundaries of the minority classes. Recently, researchers
have investigated the potential of supervised contrastive learning for
long-tailed recognition, and demonstrated that it provides a strong performance
gain. In this paper, we show that while supervised contrastive learning can
help improve performance, past baselines suffer from poor uniformity brought in
by imbalanced data distribution. This poor uniformity manifests in samples from
the minority class having poor separability in the feature space. To address
this problem, we propose targeted supervised contrastive learning (TSC), which
improves the uniformity of the feature distribution on the hypersphere. TSC
first generates a set of targets uniformly distributed on a hypersphere. It
then makes the features of different classes converge to these distinct and
uniformly distributed targets during training. This forces all classes,
including minority classes, to maintain a uniform distribution in the feature
space, improves class boundaries, and provides better generalization even in
the presence of long-tail data. Experiments on multiple datasets show that TSC
achieves state-of-the-art performance on long-tailed recognition tasks.
Related papers
- Distribution-Aware Robust Learning from Long-Tailed Data with Noisy Labels [8.14255560923536]
Real-world data often exhibit long-tailed distributions and label noise, significantly degrading generalization performance.
Recent studies have focused on noisy sample selection methods that estimate the centroid of each class based on high-confidence samples within each target class.
We present Distribution-aware Sample Selection and Contrastive Learning (DaSC) to generate enhanced class centroids.
arXiv Detail & Related papers (2024-07-23T19:06:15Z) - Probabilistic Contrastive Learning for Long-Tailed Visual Recognition [78.70453964041718]
Longtailed distributions frequently emerge in real-world data, where a large number of minority categories contain a limited number of samples.
Recent investigations have revealed that supervised contrastive learning exhibits promising potential in alleviating the data imbalance.
We propose a novel probabilistic contrastive (ProCo) learning algorithm that estimates the data distribution of the samples from each class in the feature space.
arXiv Detail & Related papers (2024-03-11T13:44:49Z) - Uncertainty-guided Boundary Learning for Imbalanced Social Event
Detection [64.4350027428928]
We propose a novel uncertainty-guided class imbalance learning framework for imbalanced social event detection tasks.
Our model significantly improves social event representation and classification tasks in almost all classes, especially those uncertain ones.
arXiv Detail & Related papers (2023-10-30T03:32:04Z) - Combating Representation Learning Disparity with Geometric Harmonization [50.29859682439571]
We propose a novel Geometric Harmonization (GH) method to encourage category-level uniformity in representation learning.
Our proposal does not alter the setting of SSL and can be easily integrated into existing methods in a low-cost manner.
arXiv Detail & Related papers (2023-10-26T17:41:11Z) - Class-Balancing Diffusion Models [57.38599989220613]
Class-Balancing Diffusion Models (CBDM) are trained with a distribution adjustment regularizer as a solution.
Our method benchmarked the generation results on CIFAR100/CIFAR100LT dataset and shows outstanding performance on the downstream recognition task.
arXiv Detail & Related papers (2023-04-30T20:00:14Z) - Class-Aware Universum Inspired Re-Balance Learning for Long-Tailed
Recognition [24.35287225775304]
Class-aware Universum Inspired Re-balance Learning(CaUIRL) for long-tailed recognition.
We develop a higher-order mixup approach, which can automatically generate class-aware Universum(CaU) data without resorting to any external data.
arXiv Detail & Related papers (2022-07-26T11:03:39Z) - Improving Tail-Class Representation with Centroid Contrastive Learning [145.73991900239017]
We propose interpolative centroid contrastive learning (ICCL) to improve long-tailed representation learning.
ICCL interpolates two images from a class-agnostic sampler and a class-aware sampler, and trains the model such that the representation of the ICCL can be used to retrieve the centroids for both source classes.
Our result shows a significant accuracy gain of 2.8% on the iNaturalist 2018 dataset with a real-world long-tailed distribution.
arXiv Detail & Related papers (2021-10-19T15:24:48Z) - Imbalanced Data Learning by Minority Class Augmentation using Capsule
Adversarial Networks [31.073558420480964]
We propose a method to restore the balance in imbalanced images, by coalescing two concurrent methods.
In our model, generative and discriminative networks play a novel competitive game.
The coalescing of capsule-GAN is effective at recognizing highly overlapping classes with much fewer parameters compared with the convolutional-GAN.
arXiv Detail & Related papers (2020-04-05T12:36:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.