Revisiting adversarial training for the worst-performing class

• URL: http://arxiv.org/abs/2302.08872v1
• Date: Fri, 17 Feb 2023 13:41:40 GMT
• Title: Revisiting adversarial training for the worst-performing class
• Authors: Thomas Pethick, Grigorios G. Chrysos, Volkan Cevher
• Abstract summary: There is a substantial gap between the top-performing and worst-performing classes in many datasets. We argue that this gap can be reduced by explicitly optimizing for the worst-performing class. Our method, called class focused online learning (CFOL), includes high probability convergence guarantees for the worst class loss.
• Score: 60.231877895663956
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite progress in adversarial training (AT), there is a substantial gap between the top-performing and worst-performing classes in many datasets. For example, on CIFAR10, the accuracies for the best and worst classes are 74% and 23%, respectively. We argue that this gap can be reduced by explicitly optimizing for the worst-performing class, resulting in a min-max-max optimization formulation. Our method, called class focused online learning (CFOL), includes high probability convergence guarantees for the worst class loss and can be easily integrated into existing training setups with minimal computational overhead. We demonstrate an improvement to 32% in the worst class accuracy on CIFAR10, and we observe consistent behavior across CIFAR100 and STL10. Our study highlights the importance of moving beyond average accuracy, which is particularly important in safety-critical applications.

Related papers

• Let the Fuzzy Rule Speak: Enhancing In-context Learning Debiasing with Interpretability [12.287692969438169]
  Large language models (LLMs) often struggle with balanced class accuracy in text classification tasks using in-context learning (ICL) This paper delves deeper into the class accuracy imbalance issue, identifying that it arises because certain classes consistently receive disproportionately high ICL probabilities. We introduce FuRud, a method for sample-level class probability correction.
  arXiv  Detail & Related papers  (2024-12-26T01:56:42Z)
• COBias and Debias: Balancing Class Accuracies for Language Models in Inference Time via Nonlinear Integer Programming [12.287692969438169]
  This paper investigates a fundamental inference-time problem in language models: imbalanced class accuracies. We find what's underneath the issue is a tendency to over-predict some classes while under-predicting some others. We show it can be effectively mitigated via inference-time optimization.
  arXiv  Detail & Related papers  (2024-05-13T10:30:33Z)
• Understanding the Detrimental Class-level Effects of Data Augmentation [63.1733767714073]
  achieving optimal average accuracy comes at the cost of significantly hurting individual class accuracy by as much as 20% on ImageNet. We present a framework for understanding how DA interacts with class-level learning dynamics. We show that simple class-conditional augmentation strategies improve performance on the negatively affected classes.
  arXiv  Detail & Related papers  (2023-12-07T18:37:43Z)
• Online Continual Learning via Logit Adjusted Softmax [24.327176079085703]
  Inter-class imbalance during training has been identified as a major cause of forgetting. We present a simple adjustment of model logits during training can effectively resist prior class bias. Our proposed method, Logit Adjusted Softmax, can mitigate the impact of inter-class imbalance not only in class-incremental but also in realistic general setups.
  arXiv  Detail & Related papers  (2023-11-11T03:03:33Z)
• Improving Robust Fairness via Balance Adversarial Training [51.67643171193376]
  Adversarial training (AT) methods are effective against adversarial attacks, yet they introduce severe disparity of accuracy and robustness between different classes. We propose Adversarial Training (BAT) to address the robust fairness problem.
  arXiv  Detail & Related papers  (2022-09-15T14:44:48Z)
• Robust Distillation for Worst-class Performance [38.80008602644002]
  We develop distillation techniques that are tailored to improve the student's worst-class performance. We show empirically that our robust distillation techniques achieve better worst-class performance. We provide insights into what makes a good teacher when the goal is to train a robust student.
  arXiv  Detail & Related papers  (2022-06-13T21:17:00Z)
• Probabilistically Robust Learning: Balancing Average- and Worst-case Performance [105.87195436925722]
  We propose a framework called robustness probabilistic that bridges the gap between the accurate, yet brittle average case and the robust, yet conservative worst case. From a theoretical point of view, this framework overcomes the trade-offs between the performance and the sample-complexity of worst-case and average-case learning.
  arXiv  Detail & Related papers  (2022-02-02T17:01:38Z)
• Learning with Multiclass AUC: Theory and Algorithms [141.63211412386283]
  Area under the ROC curve (AUC) is a well-known ranking metric for problems such as imbalanced learning and recommender systems. In this paper, we start an early trial to consider the problem of learning multiclass scoring functions via optimizing multiclass AUC metrics.
  arXiv  Detail & Related papers  (2021-07-28T05:18:10Z)
• Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning [134.15174177472807]
  We introduce adversarial training into self-supervision, to provide general-purpose robust pre-trained models for the first time. We conduct extensive experiments to demonstrate that the proposed framework achieves large performance margins.
  arXiv  Detail & Related papers  (2020-03-28T18:28:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.