Disentangling Label Distribution for Long-tailed Visual Recognition

• URL: http://arxiv.org/abs/2012.00321v2
• Date: Sat, 20 Mar 2021 15:22:19 GMT
• Title: Disentangling Label Distribution for Long-tailed Visual Recognition
• Authors: Youngkyu Hong, Seungju Han, Kwanghee Choi, Seokjun Seo, Beomsu Kim, Buru Chang
• Abstract summary: We formulate long-tailed visual recognition as a label shift problem where the target and source label distributions are different. One of the significant hurdles in dealing with the label shift problem is the entanglement between the source label distribution and the model prediction. We propose a novel method, LAbel distribution DisEntangling (LADE) loss based on the optimal bound of Donsker-Varadhan representation.
• Score: 8.538887358164438
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The current evaluation protocol of long-tailed visual recognition trains the classification model on the long-tailed source label distribution and evaluates its performance on the uniform target label distribution. Such protocol has questionable practicality since the target may also be long-tailed. Therefore, we formulate long-tailed visual recognition as a label shift problem where the target and source label distributions are different. One of the significant hurdles in dealing with the label shift problem is the entanglement between the source label distribution and the model prediction. In this paper, we focus on disentangling the source label distribution from the model prediction. We first introduce a simple but overlooked baseline method that matches the target label distribution by post-processing the model prediction trained by the cross-entropy loss and the Softmax function. Although this method surpasses state-of-the-art methods on benchmark datasets, it can be further improved by directly disentangling the source label distribution from the model prediction in the training phase. Thus, we propose a novel method, LAbel distribution DisEntangling (LADE) loss based on the optimal bound of Donsker-Varadhan representation. LADE achieves state-of-the-art performance on benchmark datasets such as CIFAR-100-LT, Places-LT, ImageNet-LT, and iNaturalist 2018. Moreover, LADE outperforms existing methods on various shifted target label distributions, showing the general adaptability of our proposed method.

Related papers

• Label Distribution Learning with Biased Annotations by Learning Multi-Label Representation [120.97262070068224]
  Multi-label learning (MLL) has gained attention for its ability to represent real-world data. Label Distribution Learning (LDL) faces challenges in collecting accurate label distributions.
  arXiv  Detail & Related papers  (2025-02-03T09:04:03Z)
• Label Distribution Learning using the Squared Neural Family on the Probability Simplex [15.680835401104247]
  We estimate a probability distribution of all possible label distributions over the simplex. With the modeled distribution, label distribution prediction can be achieved by performing the expectation operation. More information about the label distribution can be inferred, such as the prediction reliability and uncertainties.
  arXiv  Detail & Related papers  (2024-12-10T09:12:02Z)
• Theory-inspired Label Shift Adaptation via Aligned Distribution Mixture [21.494268411607766]
  We propose an innovative label shift framework named as Aligned Distribution Mixture (ADM) Within this framework, we enhance four typical label shift methods by introducing modifications to the classifier training process. Considering the distinctiveness of the proposed one-step approach, we develop an efficient bi-level optimization strategy.
  arXiv  Detail & Related papers  (2024-11-04T12:51:57Z)
• Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition [50.61991746981703]
  Current state-of-the-art LTSSL approaches rely on high-quality pseudo-labels for large-scale unlabeled data. This paper introduces a novel probabilistic framework that unifies various recent proposals in long-tail learning. We introduce a continuous contrastive learning method, CCL, extending our framework to unlabeled data using reliable and smoothed pseudo-labels.
  arXiv  Detail & Related papers  (2024-10-08T15:06:10Z)
• Exploring Beyond Logits: Hierarchical Dynamic Labeling Based on Embeddings for Semi-Supervised Classification [49.09505771145326]
  We propose a Hierarchical Dynamic Labeling (HDL) algorithm that does not depend on model predictions and utilizes image embeddings to generate sample labels. Our approach has the potential to change the paradigm of pseudo-label generation in semi-supervised learning.
  arXiv  Detail & Related papers  (2024-04-26T06:00:27Z)
• All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation [67.30502812804271]
  Pseudo-labels are widely employed in weakly supervised 3D segmentation tasks where only sparse ground-truth labels are available for learning. We propose a novel learning strategy to regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions.
  arXiv  Detail & Related papers  (2023-05-25T08:19:31Z)
• Label Distribution Learning from Logical Label [19.632157794117553]
  Label distribution learning (LDL) is an effective method to predict the label description degree (a.k.a. label distribution) of a sample. But annotating label distribution for training samples is extremely costly. We propose a novel method to learn an LDL model directly from the logical label, which unifies LE and LDL into a joint model.
  arXiv  Detail & Related papers  (2023-03-13T04:31:35Z)
• Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective [89.5370481649529]
  We propose a label distribution perspective for PU learning in this paper. Motivated by this, we propose to pursue the label distribution consistency between predicted and ground-truth label distributions. Experiments on three benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-12-06T07:38:29Z)
• Label distribution learning via label correlation grid [9.340734188957727]
  We propose a textbfLabel textbfCorrelation textbfGrid (LCG) to model the uncertainty of label relationships. Our network learns the LCG to accurately estimate the label distribution for each instance.
  arXiv  Detail & Related papers  (2022-10-15T03:58:15Z)
• Instance-Dependent Partial Label Learning [69.49681837908511]
  Partial label learning is a typical weakly supervised learning problem. Most existing approaches assume that the incorrect labels in each training example are randomly picked as the candidate labels. In this paper, we consider instance-dependent and assume that each example is associated with a latent label distribution constituted by the real number of each label.
  arXiv  Detail & Related papers  (2021-10-25T12:50:26Z)
• Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation [30.688753736660725]
  We present a simple and yet effective Distribution Alignment and Random Sampling (DARS) method to produce unbiased pseudo labels. Our method performs favorably in comparison with state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-07-23T14:45:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.