ACE and Diverse Generalization via Selective Disagreement

• URL: http://arxiv.org/abs/2509.07955v1
• Date: Tue, 09 Sep 2025 17:43:05 GMT
• Title: ACE and Diverse Generalization via Selective Disagreement
• Abstract summary: Deep neural networks are notoriously sensitive to spurious correlations.<n>Existing work on spurious correlations has often focused on incomplete correlations,leveraging access to labeled instances that break the correlation.<n>We propose learning a set of concepts that are consistent with training data but make distinct predictions on a subset of novel unlabeled inputs.
• Score: 38.64694396230853
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Deep neural networks are notoriously sensitive to spurious correlations - where a model learns a shortcut that fails out-of-distribution. Existing work on spurious correlations has often focused on incomplete correlations,leveraging access to labeled instances that break the correlation. But in cases where the spurious correlations are complete, the correct generalization is fundamentally \textit{underspecified}. To resolve this underspecification, we propose learning a set of concepts that are consistent with training data but make distinct predictions on a subset of novel unlabeled inputs. Using a self-training approach that encourages \textit{confident} and \textit{selective} disagreement, our method ACE matches or outperforms existing methods on a suite of complete-spurious correlation benchmarks, while remaining robust to incomplete spurious correlations. ACE is also more configurable than prior approaches, allowing for straight-forward encoding of prior knowledge and principled unsupervised model selection. In an early application to language-model alignment, we find that ACE achieves competitive performance on the measurement tampering detection benchmark \textit{without} access to untrusted measurements. While still subject to important limitations, ACE represents significant progress towards overcoming underspecification.

Related papers

• Beyond Hungarian: Match-Free Supervision for End-to-End Object Detection [6.786987355161583]
  We propose a novel matching-free training scheme for DETR-based detectors.<n>We use encoded ground-truth information to probe the decoder queries through a cross-attention mechanism.<n> Experimental results demonstrate that our proposed method bypasses the traditional matching process.
  arXiv  Detail & Related papers  (2026-03-09T15:44:23Z)
• Robust Molecular Property Prediction via Densifying Scarce Labeled Data [53.24886143129006]
  In drug discovery, compounds most critical for advancing research often lie beyond the training set.<n>We propose a novel bilevel optimization approach that leverages unlabeled data to interpolate between in-distribution (ID) and out-of-distribution (OOD) data.
  arXiv  Detail & Related papers  (2025-06-13T15:27:40Z)
• Improving Group Robustness on Spurious Correlation via Evidential Alignment [26.544938760265136]
  Deep neural networks often learn and rely on spurious correlations, i.e., superficial associations between non-causal features and the targets.<n>Existing methods typically mitigate this issue by using external group annotations or auxiliary deterministic models.<n>We propose Evidential Alignment, a novel framework that leverages uncertainty quantification to understand the behavior of the biased models.
  arXiv  Detail & Related papers  (2025-06-12T22:47:21Z)
• Relational Conformal Prediction for Correlated Time Series [56.59852921638328]
  We address the problem of uncertainty quantification in time series by exploiting correlated sequences.<n>We propose a novel distribution-free approach based on conformal prediction framework and quantile regression.<n>Our approach provides accurate coverage and achieves state-of-the-art uncertainty quantification in relevant benchmarks.
  arXiv  Detail & Related papers  (2025-02-13T16:12:17Z)
• Spuriousness-Aware Meta-Learning for Learning Robust Classifiers [26.544938760265136]
  Spurious correlations are brittle associations between certain attributes of inputs and target variables. Deep image classifiers often leverage them for predictions, leading to poor generalization on the data where the correlations do not hold. Mitigating the impact of spurious correlations is crucial towards robust model generalization, but it often requires annotations of the spurious correlations in data.
  arXiv  Detail & Related papers  (2024-06-15T21:41:25Z)
• Learning Robust Classifiers with Self-Guided Spurious Correlation Mitigation [26.544938760265136]
  Deep neural classifiers rely on spurious correlations between spurious attributes of inputs and targets to make predictions. We propose a self-guided spurious correlation mitigation framework. We show that training the classifier to distinguish different prediction behaviors reduces its reliance on spurious correlations without knowing them a priori.
  arXiv  Detail & Related papers  (2024-05-06T17:12:21Z)
• Unsupervised Concept Discovery Mitigates Spurious Correlations [45.48778210340187]
  Models prone to spurious correlations in training data often produce brittle predictions and introduce unintended biases. In this paper, we establish a novel connection between unsupervised object-centric learning and mitigation of spurious correlations. We introduce CoBalT: a concept balancing technique that effectively mitigates spurious correlations without requiring human labeling of subgroups.
  arXiv  Detail & Related papers  (2024-02-20T20:48:00Z)
• Open-Vocabulary Segmentation with Semantic-Assisted Calibration [68.41025728960176]
  We study open-vocabulary segmentation (OVS) through calibrating in-vocabulary and domain-biased embedding space with contextual prior of CLIP.<n>We present a Semantic-assisted CAlibration Network (SCAN) to achieve state-of-the-art performance on open-vocabulary segmentation benchmarks.
  arXiv  Detail & Related papers  (2023-12-07T07:00:09Z)
• Coherent Entity Disambiguation via Modeling Topic and Categorical Dependency [87.16283281290053]
  Previous entity disambiguation (ED) methods adopt a discriminative paradigm, where prediction is made based on matching scores between mention context and candidate entities. We propose CoherentED, an ED system equipped with novel designs aimed at enhancing the coherence of entity predictions. We achieve new state-of-the-art results on popular ED benchmarks, with an average improvement of 1.3 F1 points.
  arXiv  Detail & Related papers  (2023-11-06T16:40:13Z)
• Open Set Relation Extraction via Unknown-Aware Training [72.10462476890784]
  We propose an unknown-aware training method, regularizing the model by dynamically synthesizing negative instances. Inspired by text adversarial attacks, we adaptively apply small but critical perturbations to original training instances. Experimental results show that this method achieves SOTA unknown relation detection without compromising the classification of known relations.
  arXiv  Detail & Related papers  (2023-06-08T05:45:25Z)
• Understanding and Mitigating Spurious Correlations in Text Classification with Neighborhood Analysis [69.07674653828565]
  Machine learning models have a tendency to leverage spurious correlations that exist in the training set but may not hold true in general circumstances. In this paper, we examine the implications of spurious correlations through a novel perspective called neighborhood analysis. We propose a family of regularization methods, NFL (doN't Forget your Language) to mitigate spurious correlations in text classification.
  arXiv  Detail & Related papers  (2023-05-23T03:55:50Z)
• Look to the Right: Mitigating Relative Position Bias in Extractive Question Answering [38.36299280464046]
  Extractive question answering (QA) models tend to exploit spurious correlations to make predictions. Relative position of an answer can be exploited by QA models as superficial cues for making predictions. We propose an ensemble-based debiasing method that does not require prior knowledge about the distribution of relative positions.
  arXiv  Detail & Related papers  (2022-10-26T08:01:38Z)
• Decorrelated Clustering with Data Selection Bias [55.91842043124102]
  We propose a novel Decorrelation regularized K-Means algorithm (DCKM) for clustering with data selection bias. Our DCKM algorithm achieves significant performance gains, indicating the necessity of removing unexpected feature correlations induced by selection bias.
  arXiv  Detail & Related papers  (2020-06-29T08:55:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.