Spread Spurious Attribute: Improving Worst-group Accuracy with Spurious Attribute Estimation

• URL: http://arxiv.org/abs/2204.02070v1
• Date: Tue, 5 Apr 2022 09:08:30 GMT
• Title: Spread Spurious Attribute: Improving Worst-group Accuracy with Spurious Attribute Estimation
• Authors: Junhyun Nam, Jaehyung Kim, Jaeho Lee, Jinwoo Shin
• Abstract summary: We propose a pseudo-attribute-based algorithm, coined Spread Spurious Attribute, for improving the worst-group accuracy. Our experiments on various benchmark datasets show that our algorithm consistently outperforms the baseline methods. We also demonstrate that the proposed SSA can achieve comparable performances to methods using full (100%) spurious attribute supervision.
• Score: 72.92329724600631
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The paradigm of worst-group loss minimization has shown its promise in avoiding to learn spurious correlations, but requires costly additional supervision on spurious attributes. To resolve this, recent works focus on developing weaker forms of supervision -- e.g., hyperparameters discovered with a small number of validation samples with spurious attribute annotation -- but none of the methods retain comparable performance to methods using full supervision on the spurious attribute. In this paper, instead of searching for weaker supervisions, we ask: Given access to a fixed number of samples with spurious attribute annotations, what is the best achievable worst-group loss if we "fully exploit" them? To this end, we propose a pseudo-attribute-based algorithm, coined Spread Spurious Attribute (SSA), for improving the worst-group accuracy. In particular, we leverage samples both with and without spurious attribute annotations to train a model to predict the spurious attribute, then use the pseudo-attribute predicted by the trained model as supervision on the spurious attribute to train a new robust model having minimal worst-group loss. Our experiments on various benchmark datasets show that our algorithm consistently outperforms the baseline methods using the same number of validation samples with spurious attribute annotations. We also demonstrate that the proposed SSA can achieve comparable performances to methods using full (100%) spurious attribute supervision, by using a much smaller number of annotated samples -- from 0.6% and up to 1.5%, depending on the dataset.

Related papers

• Trained Models Tell Us How to Make Them Robust to Spurious Correlation without Group Annotation [3.894771553698554]
  Empirical Risk Minimization (ERM) models tend to rely on attributes that have high spurious correlation with the target. This can degrade the performance on underrepresented (or'minority') groups that lack these attributes. We propose Environment-based Validation and Loss-based Sampling (EVaLS) to enhance robustness to spurious correlation.
  arXiv  Detail & Related papers  (2024-10-07T08:17:44Z)
• Task-oriented Embedding Counts: Heuristic Clustering-driven Feature Fine-tuning for Whole Slide Image Classification [1.292108130501585]
  We propose a clustering-driven feature fine-tuning method (HC-FT) to enhance the performance of multiple instance learning. The proposed method is evaluated on both CAMELYON16 and BRACS datasets, achieving an AUC of 97.13% and 85.85%, respectively.
  arXiv  Detail & Related papers  (2024-06-02T08:53:45Z)
• Annotation-Free Group Robustness via Loss-Based Resampling [3.355491272942994]
  Training neural networks for image classification with empirical risk minimization makes them vulnerable to relying on spurious attributes instead of causal ones for prediction. We propose a new method, called loss-based feature re-weighting (LFR), in which we infer a grouping of the data by evaluating an ERM-pre-trained model on a small left-out split of the training data. For a complete assessment, we evaluate LFR on various versions of Waterbirds and CelebA datasets with different spurious correlations.
  arXiv  Detail & Related papers  (2023-12-08T08:22:02Z)
• Multi-Group Fairness Evaluation via Conditional Value-at-Risk Testing [24.553384023323332]
  We propose an approach to test for performance disparities based on Conditional Value-at-Risk. We show that the sample complexity required for discovering performance violations is reduced exponentially to be at most upper bounded by the square root of the number of groups.
  arXiv  Detail & Related papers  (2023-12-06T19:25:32Z)
• Causal Effect Regularization: Automated Detection and Removal of Spurious Attributes [13.852987916253685]
  In many classification datasets, the task labels are spuriously correlated with some input attributes. We propose a method to automatically identify spurious attributes by estimating their causal effect on the label. Our method mitigates the reliance on spurious attributes even under noisy estimation of causal effects.
  arXiv  Detail & Related papers  (2023-06-19T17:17:42Z)
• Breaking the Spurious Causality of Conditional Generation via Fairness Intervention with Corrective Sampling [77.15766509677348]
  Conditional generative models often inherit spurious correlations from the training dataset. This can result in label-conditional distributions that are imbalanced with respect to another latent attribute. We propose a general two-step strategy to mitigate this issue.
  arXiv  Detail & Related papers  (2022-12-05T08:09:33Z)
• MACSum: Controllable Summarization with Mixed Attributes [56.685735509260276]
  MACSum is the first human-annotated summarization dataset for controlling mixed attributes. We propose two simple and effective parameter-efficient approaches for the new task of mixed controllable summarization.
  arXiv  Detail & Related papers  (2022-11-09T17:17:37Z)
• Fairness via Adversarial Attribute Neighbourhood Robust Learning [49.93775302674591]
  We propose a principled underlineRobust underlineAdversarial underlineAttribute underlineNeighbourhood (RAAN) loss to debias the classification head.
  arXiv  Detail & Related papers  (2022-10-12T23:39:28Z)
• WSSOD: A New Pipeline for Weakly- and Semi-Supervised Object Detection [75.80075054706079]
  We propose a weakly- and semi-supervised object detection framework (WSSOD) An agent detector is first trained on a joint dataset and then used to predict pseudo bounding boxes on weakly-annotated images. The proposed framework demonstrates remarkable performance on PASCAL-VOC and MSCOCO benchmark, achieving a high performance comparable to those obtained in fully-supervised settings.
  arXiv  Detail & Related papers  (2021-05-21T11:58:50Z)
• Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification [137.9939571408506]
  We estimate and exploit the credibility of the assigned pseudo-label of each sample to alleviate the influence of noisy labels. Our uncertainty-guided optimization brings significant improvement and achieves the state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-16T04:09:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.