BARACK: Partially Supervised Group Robustness With Guarantees

• URL: http://arxiv.org/abs/2201.00072v1
• Date: Fri, 31 Dec 2021 23:05:21 GMT
• Title: BARACK: Partially Supervised Group Robustness With Guarantees
• Authors: Nimit Sohoni, Maziar Sanjabi, Nicolas Ballas, Aditya Grover, Shaoliang Nie, Hamed Firooz, Christopher R\'e
• Abstract summary: We propose BARACK, a framework to improve worst-group performance on neural networks. We train a model to predict the missing group labels for the training data, and then use these predicted group labels in a robust optimization objective. Empirically, our method outperforms the baselines that do not use group information, even when only 1-33% of points have group labels.
• Score: 29.427365308680717
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While neural networks have shown remarkable success on classification tasks in terms of average-case performance, they often fail to perform well on certain groups of the data. Such group information may be expensive to obtain; thus, recent works in robustness and fairness have proposed ways to improve worst-group performance even when group labels are unavailable for the training data. However, these methods generally underperform methods that utilize group information at training time. In this work, we assume access to a small number of group labels alongside a larger dataset without group labels. We propose BARACK, a simple two-step framework to utilize this partial group information to improve worst-group performance: train a model to predict the missing group labels for the training data, and then use these predicted group labels in a robust optimization objective. Theoretically, we provide generalization bounds for our approach in terms of the worst-group performance, showing how the generalization error scales with respect to both the total number of training points and the number of training points with group labels. Empirically, our method outperforms the baselines that do not use group information, even when only 1-33% of points have group labels. We provide ablation studies to support the robustness and extensibility of our framework.

Related papers

• Data Debiasing with Datamodels (D3M): Improving Subgroup Robustness via Data Selection [80.85902083005237]
  We introduce Data Debiasing with Datamodels (D3M), a debiasing approach which isolates and removes specific training examples that drive the model's failures on minority groups.
  arXiv  Detail & Related papers  (2024-06-24T17:51:01Z)
• Outlier-Robust Group Inference via Gradient Space Clustering [50.87474101594732]
  Existing methods can improve the worst-group performance, but they require group annotations, which are often expensive and sometimes infeasible to obtain. We address the problem of learning group annotations in the presence of outliers by clustering the data in the space of gradients of the model parameters. We show that data in the gradient space has a simpler structure while preserving information about minority groups and outliers, making it suitable for standard clustering methods like DBSCAN.
  arXiv  Detail & Related papers  (2022-10-13T06:04:43Z)
• Take One Gram of Neural Features, Get Enhanced Group Robustness [23.541213868620837]
  Predictive performance of machine learning models trained with empirical risk minimization can degrade considerably under distribution shifts. We propose to partition the training dataset into groups based on Gram matrices of features extracted by an identification'' model. Our approach not only improves group robustness over ERM but also outperforms all recent baselines.
  arXiv  Detail & Related papers  (2022-08-26T12:34:55Z)
• Improved Group Robustness via Classifier Retraining on Independent Splits [6.930560177764658]
  Group distributionally robust optimization is a widely used baseline for learning models with strong worst-group performance. This paper designs a simple method based on the idea of retraining on independent splits of the training data. We find that using a novel sample-splitting procedure achieves robust worst-group performance in the fine-tuning step.
  arXiv  Detail & Related papers  (2022-04-20T16:22:27Z)
• Towards Group Robustness in the presence of Partial Group Labels [61.33713547766866]
  spurious correlations between input samples and the target labels wrongly direct the neural network predictions. We propose an algorithm that optimize for the worst-off group assignments from a constraint set. We show improvements in the minority group's performance while preserving overall aggregate accuracy across groups.
  arXiv  Detail & Related papers  (2022-01-10T22:04:48Z)
• Focus on the Common Good: Group Distributional Robustness Follows [47.62596240492509]
  This paper proposes a new and simple algorithm that explicitly encourages learning of features that are shared across various groups. While Group-DRO focuses on groups with worst regularized loss, focusing instead, on groups that enable better performance even on other groups, could lead to learning of shared/common features.
  arXiv  Detail & Related papers  (2021-10-06T09:47:41Z)
• Just Train Twice: Improving Group Robustness without Training Group Information [101.84574184298006]
  Standard training via empirical risk minimization can produce models that achieve high accuracy on average but low accuracy on certain groups. Prior approaches that achieve high worst-group accuracy, like group distributionally robust optimization (group DRO) require expensive group annotations for each training point. We propose a simple two-stage approach, JTT, that first trains a standard ERM model for several epochs, and then trains a second model that upweights the training examples that the first model misclassified.
  arXiv  Detail & Related papers  (2021-07-19T17:52:32Z)
• Social Adaptive Module for Weakly-supervised Group Activity Recognition [143.68241396839062]
  This paper presents a new task named weakly-supervised group activity recognition (GAR) It differs from conventional GAR tasks in that only video-level labels are available, yet the important persons within each frame are not provided even in the training data. This eases us to collect and annotate a large-scale NBA dataset and thus raise new challenges to GAR.
  arXiv  Detail & Related papers  (2020-07-18T16:40:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.