Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification

• URL: http://arxiv.org/abs/2112.14380v1
• Date: Wed, 29 Dec 2021 03:18:47 GMT
• Title: Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification
• Authors: Beier Zhu, Yulei Niu, Xian-Sheng Hua, Hanwang Zhang
• Abstract summary: We address the overlooked unbiasedness in existing long-tailed classification methods. We propose Cross-Domain Empirical Risk Minimization (xERM) for training an unbiased model.
• Score: 90.17537630880305
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We address the overlooked unbiasedness in existing long-tailed classification methods: we find that their overall improvement is mostly attributed to the biased preference of tail over head, as the test distribution is assumed to be balanced; however, when the test is as imbalanced as the long-tailed training data -- let the test respect Zipf's law of nature -- the tail bias is no longer beneficial overall because it hurts the head majorities. In this paper, we propose Cross-Domain Empirical Risk Minimization (xERM) for training an unbiased model to achieve strong performances on both test distributions, which empirically demonstrates that xERM fundamentally improves the classification by learning better feature representation rather than the head vs. tail game. Based on causality, we further theoretically explain why xERM achieves unbiasedness: the bias caused by the domain selection is removed by adjusting the empirical risks on the imbalanced domain and the balanced but unseen domain. Codes are available at https://github.com/BeierZhu/xERM.

Related papers

• Revisiting the Dataset Bias Problem from a Statistical Perspective [72.94990819287551]
  We study the "dataset bias" problem from a statistical standpoint. We identify the main cause of the problem as the strong correlation between a class attribute u and a non-class attribute b. We propose to mitigate dataset bias via either weighting the objective of each sample n by frac1p(u_n|b_n) or sampling that sample with a weight proportional to frac1p(u_n|b_n).
  arXiv  Detail & Related papers  (2024-02-05T22:58:06Z)
• Rectify the Regression Bias in Long-Tailed Object Detection [29.34827806854778]
  Long-tailed object detection faces great challenges because of its extremely imbalanced class distribution. Recent methods mainly focus on the classification bias and its loss function design, while ignoring the subtle influence of the regression branch. This paper shows that the regression bias exists and does adversely and seriously impact the detection accuracy.
  arXiv  Detail & Related papers  (2024-01-29T04:40:33Z)
• Marginal Debiased Network for Fair Visual Recognition [59.05212866862219]
  We propose a novel marginal debiased network (MDN) to learn debiased representations. Our MDN can achieve a remarkable performance on under-represented samples.
  arXiv  Detail & Related papers  (2024-01-04T08:57:09Z)
• Causality and Independence Enhancement for Biased Node Classification [56.38828085943763]
  We propose a novel Causality and Independence Enhancement (CIE) framework, applicable to various graph neural networks (GNNs) Our approach estimates causal and spurious features at the node representation level and mitigates the influence of spurious correlations. Our approach CIE not only significantly enhances the performance of GNNs but outperforms state-of-the-art debiased node classification methods.
  arXiv  Detail & Related papers  (2023-10-14T13:56:24Z)
• Dual Compensation Residual Networks for Class Imbalanced Learning [98.35401757647749]
  We propose Dual Compensation Residual Networks to better fit both tail and head classes. An important factor causing overfitting is that there is severe feature drift between training and test data on tail classes. We also propose a Residual Balanced Multi-Proxies classifier to alleviate the under-fitting issue.
  arXiv  Detail & Related papers  (2023-08-25T04:06:30Z)
• Pseudo Bias-Balanced Learning for Debiased Chest X-ray Classification [57.53567756716656]
  We study the problem of developing debiased chest X-ray diagnosis models without knowing exactly the bias labels. We propose a novel algorithm, pseudo bias-balanced learning, which first captures and predicts per-sample bias labels. Our proposed method achieved consistent improvements over other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-18T11:02:18Z)
• Towards Calibrated Model for Long-Tailed Visual Recognition from Prior Perspective [17.733087434470907]
  Real-world data universally confronts a severe class-imbalance problem and exhibits a long-tailed distribution. We propose two novel methods from the prior perspective to alleviate this dilemma. First, we deduce a balance-oriented data augmentation named Uniform Mixup (UniMix) to promote mixup in long-tailed scenarios. Second, motivated by the Bayesian theory, we figure out the Bayes Bias (Bayias) to compensate it as a modification on standard cross-entropy loss.
  arXiv  Detail & Related papers  (2021-11-06T12:53:34Z)
• Debiasing classifiers: is reality at variance with expectation? [9.730485257882433]
  We show that debiasers often fail in practice to generalize out-of-sample data, and can in fact make fairness worse rather than better. Considering fairness--performance trade-offs justifies the counterintuitive notion that partial debiasing can actually yield better results in practice on out-of-sample data.
  arXiv  Detail & Related papers  (2020-11-04T17:00:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.