Importance Tempering: Group Robustness for Overparameterized Models

• URL: http://arxiv.org/abs/2209.08745v1
• Date: Mon, 19 Sep 2022 03:41:30 GMT
• Title: Importance Tempering: Group Robustness for Overparameterized Models
• Authors: Yiping Lu, Wenlong Ji, Zachary Izzo, Lexing Ying
• Abstract summary: We propose importance tempering to improve the decision boundary. We prove that properly selected temperatures can extricate the minority collapse for imbalanced classification. Empirically, we achieve state-of-the-art results on worst group classification tasks using importance tempering.
• Score: 12.559727665706687
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although overparameterized models have shown their success on many machine learning tasks, the accuracy could drop on the testing distribution that is different from the training one. This accuracy drop still limits applying machine learning in the wild. At the same time, importance weighting, a traditional technique to handle distribution shifts, has been demonstrated to have less or even no effect on overparameterized models both empirically and theoretically. In this paper, we propose importance tempering to improve the decision boundary and achieve consistently better results for overparameterized models. Theoretically, we justify that the selection of group temperature can be different under label shift and spurious correlation setting. At the same time, we also prove that properly selected temperatures can extricate the minority collapse for imbalanced classification. Empirically, we achieve state-of-the-art results on worst group classification tasks using importance tempering.

Related papers

• Counterfactual Fairness through Transforming Data Orthogonal to Bias [7.109458605736819]
  We propose a novel data pre-processing algorithm, Orthogonal to Bias (OB) OB is designed to eliminate the influence of a group of continuous sensitive variables, thus promoting counterfactual fairness in machine learning applications. OB is model-agnostic, making it applicable to a wide range of machine learning models and tasks.
  arXiv  Detail & Related papers  (2024-03-26T16:40:08Z)
• On the Limitations of Temperature Scaling for Distributions with Overlaps [8.486166869140929]
  We show that for empirical risk minimizers for a general set of distributions, the performance of temperature scaling degrades with the amount of overlap between classes. We prove that optimizing a modified form of the empirical risk induced by the Mixup data augmentation technique can in fact lead to reasonably good calibration performance.
  arXiv  Detail & Related papers  (2023-06-01T14:35:28Z)
• DualFair: Fair Representation Learning at Both Group and Individual Levels via Contrastive Self-supervision [73.80009454050858]
  This work presents a self-supervised model, called DualFair, that can debias sensitive attributes like gender and race from learned representations. Our model jointly optimize for two fairness criteria - group fairness and counterfactual fairness.
  arXiv  Detail & Related papers  (2023-03-15T07:13:54Z)
• Distributionally Robust Models with Parametric Likelihood Ratios [123.05074253513935]
  Three simple ideas allow us to train models with DRO using a broader class of parametric likelihood ratios. We find that models trained with the resulting parametric adversaries are consistently more robust to subpopulation shifts when compared to other DRO approaches.
  arXiv  Detail & Related papers  (2022-04-13T12:43:12Z)
• FairPrune: Achieving Fairness Through Pruning for Dermatological Disease Diagnosis [17.508632873527525]
  We propose a method, FairPrune, that achieves fairness by pruning. We show that our method can greatly improve fairness while keeping the average accuracy of both groups as high as possible.
  arXiv  Detail & Related papers  (2022-03-04T02:57:34Z)
• FairIF: Boosting Fairness in Deep Learning via Influence Functions with Validation Set Sensitive Attributes [51.02407217197623]
  We propose a two-stage training algorithm named FAIRIF. It minimizes the loss over the reweighted data set where the sample weights are computed. We show that FAIRIF yields models with better fairness-utility trade-offs against various types of bias.
  arXiv  Detail & Related papers  (2022-01-15T05:14:48Z)
• Automatic Mixed-Precision Quantization Search of BERT [62.65905462141319]
  Pre-trained language models such as BERT have shown remarkable effectiveness in various natural language processing tasks. These models usually contain millions of parameters, which prevents them from practical deployment on resource-constrained devices. We propose an automatic mixed-precision quantization framework designed for BERT that can simultaneously conduct quantization and pruning in a subgroup-wise level.
  arXiv  Detail & Related papers  (2021-12-30T06:32:47Z)
• Characterizing Fairness Over the Set of Good Models Under Selective Labels [69.64662540443162]
  We develop a framework for characterizing predictive fairness properties over the set of models that deliver similar overall performance. We provide tractable algorithms to compute the range of attainable group-level predictive disparities. We extend our framework to address the empirically relevant challenge of selectively labelled data.
  arXiv  Detail & Related papers  (2021-01-02T02:11:37Z)
• Selective Classification Can Magnify Disparities Across Groups [89.14499988774985]
  We find that while selective classification can improve average accuracies, it can simultaneously magnify existing accuracy disparities. Increasing abstentions can even decrease accuracies on some groups. We train distributionally-robust models that achieve similar full-coverage accuracies across groups and show that selective classification uniformly improves each group.
  arXiv  Detail & Related papers  (2020-10-27T08:51:30Z)
• Towards Threshold Invariant Fair Classification [10.317169065327546]
  This paper introduces the notion of threshold invariant fairness, which enforces equitable performances across different groups independent of the decision threshold. Experimental results demonstrate that the proposed methodology is effective to alleviate the threshold sensitivity in machine learning models designed to achieve fairness.
  arXiv  Detail & Related papers  (2020-06-18T16:49:46Z)
• Mitigating Class Boundary Label Uncertainty to Reduce Both Model Bias and Variance [4.563176550691304]
  We investigate a new approach to handle inaccuracy and uncertainty in the training data labels. Our method can reduce both bias and variance by estimating the pointwise label uncertainty of the training set.
  arXiv  Detail & Related papers  (2020-02-23T18:24:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.