Reviving Undersampling for Long-Tailed Learning

• URL: http://arxiv.org/abs/2401.16811v1
• Date: Tue, 30 Jan 2024 08:15:13 GMT
• Title: Reviving Undersampling for Long-Tailed Learning
• Authors: Hao Yu, Yingxiao Du, Jianxin Wu
• Abstract summary: We aim to enhance the accuracy of the worst-performing categories and utilize the harmonic mean and geometric mean to assess the model's performance. We devise a straightforward model ensemble strategy, which does not result in any additional overhead and achieves improved harmonic and geometric mean. We validate the effectiveness of our approach on widely utilized benchmark datasets for long-tailed learning.
• Score: 16.054442161144603
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: The training datasets used in long-tailed recognition are extremely unbalanced, resulting in significant variation in per-class accuracy across categories. Prior works mostly used average accuracy to evaluate their algorithms, which easily ignores those worst-performing categories. In this paper, we aim to enhance the accuracy of the worst-performing categories and utilize the harmonic mean and geometric mean to assess the model's performance. We revive the balanced undersampling idea to achieve this goal. In few-shot learning, balanced subsets are few-shot and will surely under-fit, hence it is not used in modern long-tailed learning. But, we find that it produces a more equitable distribution of accuracy across categories with much higher harmonic and geometric mean accuracy, and, but lower average accuracy. Moreover, we devise a straightforward model ensemble strategy, which does not result in any additional overhead and achieves improved harmonic and geometric mean while keeping the average accuracy almost intact when compared to state-of-the-art long-tailed learning methods. We validate the effectiveness of our approach on widely utilized benchmark datasets for long-tailed learning. Our code is at \href{https://github.com/yuhao318/BTM/}{https://github.com/yuhao318/BTM/}.

Related papers

• Achieving More with Less: A Tensor-Optimization-Powered Ensemble Method [53.170053108447455]
  Ensemble learning is a method that leverages weak learners to produce a strong learner. We design a smooth and convex objective function that leverages the concept of margin, making the strong learner more discriminative. We then compare our algorithm with random forests of ten times the size and other classical methods across numerous datasets.
  arXiv  Detail & Related papers  (2024-08-06T03:42:38Z)
• Understanding the Detrimental Class-level Effects of Data Augmentation [63.1733767714073]
  achieving optimal average accuracy comes at the cost of significantly hurting individual class accuracy by as much as 20% on ImageNet. We present a framework for understanding how DA interacts with class-level learning dynamics. We show that simple class-conditional augmentation strategies improve performance on the negatively affected classes.
  arXiv  Detail & Related papers  (2023-12-07T18:37:43Z)
• Unsupervised Estimation of Ensemble Accuracy [0.0]
  We present a method for estimating the joint power of several classifiers. It differs from existing approaches which focus on "diversity" measures by not relying on labels. We demonstrate the method on popular large-scale face recognition datasets.
  arXiv  Detail & Related papers  (2023-11-18T02:31:36Z)
• No One Left Behind: Improving the Worst Categories in Long-Tailed Learning [29.89394406438639]
  We argue that under such an evaluation setting, some categories are inevitably sacrificed. We propose a simple plug-in method that is applicable to a wide range of methods.
  arXiv  Detail & Related papers  (2023-03-07T03:24:54Z)
• Compound Batch Normalization for Long-tailed Image Classification [77.42829178064807]
  We propose a compound batch normalization method based on a Gaussian mixture. It can model the feature space more comprehensively and reduce the dominance of head classes. The proposed method outperforms existing methods on long-tailed image classification.
  arXiv  Detail & Related papers  (2022-12-02T07:31:39Z)
• Intra-class Adaptive Augmentation with Neighbor Correction for Deep Metric Learning [99.14132861655223]
  We propose a novel intra-class adaptive augmentation (IAA) framework for deep metric learning. We reasonably estimate intra-class variations for every class and generate adaptive synthetic samples to support hard samples mining. Our method significantly improves and outperforms the state-of-the-art methods on retrieval performances by 3%-6%.
  arXiv  Detail & Related papers  (2022-11-29T14:52:38Z)
• Worst Case Matters for Few-Shot Recognition [27.023352955311502]
  Few-shot recognition learns a recognition model with very few (e.g., 1 or 5) images per category. Current few-shot learning methods focus on improving the average accuracy over many episodes. We argue that in real-world applications we may often only try one episode instead of many, and hence maximizing the worst-case accuracy is more important than maximizing the average accuracy.
  arXiv  Detail & Related papers  (2022-03-13T05:39:40Z)
• Learning Optimal Fair Classification Trees: Trade-offs Between Interpretability, Fairness, and Accuracy [7.215903549622416]
  We propose a mixed integer optimization framework for learning optimal classification trees. We benchmark our method against state-of-the-art approaches for fair classification on popular datasets. Our method consistently finds decisions with almost full parity, while other methods rarely do.
  arXiv  Detail & Related papers  (2022-01-24T19:47:10Z)
• Newer is not always better: Rethinking transferability metrics, their peculiarities, stability and performance [5.650647159993238]
  Fine-tuning of large pre-trained image and language models on small customized datasets has become increasingly popular. We show that the statistical problems with covariance estimation drive the poor performance of H-score. We propose a correction and recommend measuring correlation performance against relative accuracy in such settings.
  arXiv  Detail & Related papers  (2021-10-13T17:24:12Z)
• Improving Calibration for Long-Tailed Recognition [68.32848696795519]
  We propose two methods to improve calibration and performance in such scenarios. For dataset bias due to different samplers, we propose shifted batch normalization. Our proposed methods set new records on multiple popular long-tailed recognition benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-01T13:55:21Z)
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift [81.74795324629712]
  We call prediction-time batch normalization, which significantly improves model accuracy and calibration under covariate shift. We show that prediction-time batch normalization provides complementary benefits to existing state-of-the-art approaches for improving robustness. The method has mixed results when used alongside pre-training, and does not seem to perform as well under more natural types of dataset shift.
  arXiv  Detail & Related papers  (2020-06-19T05:08:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.