Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect

• URL: http://arxiv.org/abs/2009.12991v4
• Date: Thu, 11 Feb 2021 04:10:13 GMT
• Title: Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect
• Authors: Kaihua Tang, Jianqiang Huang, Hanwang Zhang
• Abstract summary: Long-tailed classification is the key to deep learning at scale. Existing methods are mainly based on re-weighting/resamplings that lack a fundamental theory. In this paper, we establish a causal inference framework, which not only unravels the whys of previous methods, but also derives a new principled solution.
• Score: 95.37587481952487
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As the class size grows, maintaining a balanced dataset across many classes is challenging because the data are long-tailed in nature; it is even impossible when the sample-of-interest co-exists with each other in one collectable unit, e.g., multiple visual instances in one image. Therefore, long-tailed classification is the key to deep learning at scale. However, existing methods are mainly based on re-weighting/re-sampling heuristics that lack a fundamental theory. In this paper, we establish a causal inference framework, which not only unravels the whys of previous methods, but also derives a new principled solution. Specifically, our theory shows that the SGD momentum is essentially a confounder in long-tailed classification. On one hand, it has a harmful causal effect that misleads the tail prediction biased towards the head. On the other hand, its induced mediation also benefits the representation learning and head prediction. Our framework elegantly disentangles the paradoxical effects of the momentum, by pursuing the direct causal effect caused by an input sample. In particular, we use causal intervention in training, and counterfactual reasoning in inference, to remove the "bad" while keep the "good". We achieve new state-of-the-arts on three long-tailed visual recognition benchmarks: Long-tailed CIFAR-10/-100, ImageNet-LT for image classification and LVIS for instance segmentation.

Related papers

• Rectify the Regression Bias in Long-Tailed Object Detection [29.34827806854778]
  Long-tailed object detection faces great challenges because of its extremely imbalanced class distribution. Recent methods mainly focus on the classification bias and its loss function design, while ignoring the subtle influence of the regression branch. This paper shows that the regression bias exists and does adversely and seriously impact the detection accuracy.
  arXiv  Detail & Related papers  (2024-01-29T04:40:33Z)
• Understanding the Detrimental Class-level Effects of Data Augmentation [63.1733767714073]
  achieving optimal average accuracy comes at the cost of significantly hurting individual class accuracy by as much as 20% on ImageNet. We present a framework for understanding how DA interacts with class-level learning dynamics. We show that simple class-conditional augmentation strategies improve performance on the negatively affected classes.
  arXiv  Detail & Related papers  (2023-12-07T18:37:43Z)
• SINCERE: Supervised Information Noise-Contrastive Estimation REvisited [5.004880836963827]
  Previous work suggests a supervised contrastive (SupCon) loss to extend InfoNCE to learn from available class labels. We propose the Supervised InfoNCE REvisited (SINCERE) loss as a theoretically-justified supervised extension of InfoNCE. Experiments show that SINCERE leads to better separation of embeddings from different classes and improves transfer learning classification accuracy.
  arXiv  Detail & Related papers  (2023-09-25T16:40:56Z)
• The Equalization Losses: Gradient-Driven Training for Long-tailed Object Recognition [84.51875325962061]
  We propose a gradient-driven training mechanism to tackle the long-tail problem. We introduce a new family of gradient-driven loss functions, namely equalization losses. Our method consistently outperforms the baseline models.
  arXiv  Detail & Related papers  (2022-10-11T16:00:36Z)
• Constructing Balance from Imbalance for Long-tailed Image Recognition [50.6210415377178]
  The imbalance between majority (head) classes and minority (tail) classes severely skews the data-driven deep neural networks. Previous methods tackle with data imbalance from the viewpoints of data distribution, feature space, and model design. We propose a concise paradigm by progressively adjusting label space and dividing the head classes and tail classes. Our proposed model also provides a feature evaluation method and paves the way for long-tailed feature learning.
  arXiv  Detail & Related papers  (2022-08-04T10:22:24Z)
• Relieving Long-tailed Instance Segmentation via Pairwise Class Balance [85.53585498649252]
  Long-tailed instance segmentation is a challenging task due to the extreme imbalance of training samples among classes. It causes severe biases of the head classes (with majority samples) against the tailed ones. We propose a novel Pairwise Class Balance (PCB) method, built upon a confusion matrix which is updated during training to accumulate the ongoing prediction preferences.
  arXiv  Detail & Related papers  (2022-01-08T07:48:36Z)
• You Only Need End-to-End Training for Long-Tailed Recognition [8.789819609485225]
  Cross-entropy loss tends to produce highly correlated features on imbalanced data. We propose two novel modules, Block-based Relatively Balanced Batch Sampler (B3RS) and Batch Embedded Training (BET) Experimental results on the long-tailed classification benchmarks, CIFAR-LT and ImageNet-LT, demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2021-12-11T11:44:09Z)
• Investigate the Essence of Long-Tailed Recognition from a Unified Perspective [11.080317683184363]
  deep recognition models often suffer from long-tailed data distributions due to heavy imbalanced sample number across categories. In this work, we demonstrate that long-tailed recognition suffers from both sample number and category similarity.
  arXiv  Detail & Related papers  (2021-07-08T11:08:40Z)
• Distributional Robustness Loss for Long-tail Learning [20.800627115140465]
  Real-world data is often unbalanced and long-tailed, but deep models struggle to recognize rare classes in the presence of frequent classes. We show that the feature extractor part of deep networks suffers greatly from this bias. We propose a new loss based on robustness theory, which encourages the model to learn high-quality representations for both head and tail classes.
  arXiv  Detail & Related papers  (2021-04-07T11:34:04Z)
• Seesaw Loss for Long-Tailed Instance Segmentation [131.86306953253816]
  We propose Seesaw Loss to dynamically re-balance gradients of positive and negative samples for each category. The mitigation factor reduces punishments to tail categories w.r.t. the ratio of cumulative training instances between different categories. The compensation factor increases the penalty of misclassified instances to avoid false positives of tail categories.
  arXiv  Detail & Related papers  (2020-08-23T12:44:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.