Robust Long-Tailed Learning via Label-Aware Bounded CVaR

• URL: http://arxiv.org/abs/2308.15405v1
• Date: Tue, 29 Aug 2023 16:07:18 GMT
• Title: Robust Long-Tailed Learning via Label-Aware Bounded CVaR
• Authors: Hong Zhu, Runpeng Yu, Xing Tang, Yifei Wang, Yuan Fang, Yisen Wang
• Abstract summary: We propose two novel approaches to improve the performance of long-tailed learning with a solid theoretical ground. Specifically, we introduce a Label-Aware Bounded CVaR loss to overcome the pessimistic result of the original CVaR. We additionally propose a LAB-CVaR with logit adjustment to stabilize the optimization process.
• Score: 36.26100472960534
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data in the real-world classification problems are always imbalanced or long-tailed, wherein the majority classes have the most of the samples that dominate the model training. In such setting, the naive model tends to have poor performance on the minority classes. Previously, a variety of loss modifications have been proposed to address the long-tailed leaning problem, while these methods either treat the samples in the same class indiscriminatingly or lack a theoretical guarantee. In this paper, we propose two novel approaches based on CVaR (Conditional Value at Risk) to improve the performance of long-tailed learning with a solid theoretical ground. Specifically, we firstly introduce a Label-Aware Bounded CVaR (LAB-CVaR) loss to overcome the pessimistic result of the original CVaR, and further design the optimal weight bounds for LAB-CVaR theoretically. Based on LAB-CVaR, we additionally propose a LAB-CVaR with logit adjustment (LAB-CVaR-logit) loss to stabilize the optimization process, where we also offer the theoretical support. Extensive experiments on real-world datasets with long-tailed label distributions verify the superiority of our proposed methods.

Related papers

• DRAUC: An Instance-wise Distributionally Robust AUC Optimization Framework [133.26230331320963]
  Area Under the ROC Curve (AUC) is a widely employed metric in long-tailed classification scenarios. We propose an instance-wise surrogate loss of Distributionally Robust AUC (DRAUC) and build our optimization framework on top of it.
  arXiv  Detail & Related papers  (2023-11-06T12:15:57Z)
• TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization [89.54947228958494]
  This paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks. We propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework. TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness.
  arXiv  Detail & Related papers  (2023-03-20T14:12:55Z)
• Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
  Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data. We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
  arXiv  Detail & Related papers  (2023-03-06T14:49:59Z)
• Proposal Distribution Calibration for Few-Shot Object Detection [65.19808035019031]
  In few-shot object detection (FSOD), the two-step training paradigm is widely adopted to mitigate the severe sample imbalance. Unfortunately, the extreme data scarcity aggravates the proposal distribution bias, hindering the RoI head from evolving toward novel classes. We introduce a simple yet effective proposal distribution calibration (PDC) approach to neatly enhance the localization and classification abilities of the RoI head.
  arXiv  Detail & Related papers  (2022-12-15T05:09:11Z)
• Distributionally Robust Models with Parametric Likelihood Ratios [123.05074253513935]
  Three simple ideas allow us to train models with DRO using a broader class of parametric likelihood ratios. We find that models trained with the resulting parametric adversaries are consistently more robust to subpopulation shifts when compared to other DRO approaches.
  arXiv  Detail & Related papers  (2022-04-13T12:43:12Z)
• Boosted CVaR Classification [44.731468512484824]
  A popular approach to maximize the model's tail performance is to minimize the CVaR loss. We show that for classification tasks where models are evaluated by the zero-one loss, the minimizer of the average zero-one loss also minimizes the CVaR zero-one loss. We propose the Boosted CVaR Classification framework which is motivated by a direct relationship between CVaR and a classical boosting algorithm called LPBoost.
  arXiv  Detail & Related papers  (2021-10-26T18:27:25Z)
• Ortho-Shot: Low Displacement Rank Regularization with Data Augmentation for Few-Shot Learning [23.465747123791772]
  In few-shot classification, the primary goal is to learn representations that generalize well for novel classes. We propose an efficient low displacement rank (LDR) regularization strategy termed Ortho-Shot.
  arXiv  Detail & Related papers  (2021-10-18T14:58:36Z)
• Learning with Multiclass AUC: Theory and Algorithms [141.63211412386283]
  Area under the ROC curve (AUC) is a well-known ranking metric for problems such as imbalanced learning and recommender systems. In this paper, we start an early trial to consider the problem of learning multiclass scoring functions via optimizing multiclass AUC metrics.
  arXiv  Detail & Related papers  (2021-07-28T05:18:10Z)
• Model Rectification via Unknown Unknowns Extraction from Deployment Samples [8.0497115494227]
  We propose a general algorithmic framework that aims to perform a post-training model rectification at deployment time in a supervised way. RTSCV extracts unknown unknowns (u.u.s) We show that RTSCV consistently outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-02-08T11:46:19Z)
• Learning with CVaR-based feedback under potentially heavy tails [8.572654816871873]
  We study learning algorithms that seek to minimize the conditional value-at-risk (CVaR) We first study a general-purpose estimator of CVaR for potentially heavy-tailed random variables. We then derive a new learning algorithm which robustly chooses among candidates produced by gradient-driven sub-processes.
  arXiv  Detail & Related papers  (2020-06-03T01:08:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.