Deep Attention Reweighting: Post-Hoc Attention-Based Feature Aggregation in CNNs for Disentangling Core and Spurious Features under Spurious Correlations

• URL: http://arxiv.org/abs/2605.20732v1
• Date: Wed, 20 May 2026 05:35:45 GMT
• Title: Deep Attention Reweighting: Post-Hoc Attention-Based Feature Aggregation in CNNs for Disentangling Core and Spurious Features under Spurious Correlations
• Abstract summary: Deep Feature Reweighting (DFR) is a technique that reduces a trained model's reliance on spurious correlations by retraining its classification head on a target dataset.<n>We show that DFR is fundamentally constrained by operating on entangled features, limiting its ability to amplify the core features while simultaneously suppressing the spurious ones.<n>We propose Deep Attention Reweighting (DAR), a post-hoc attention-based aggregation module that replaces GAP and is retrained jointly with the classification head.
• Score: 2.3234871809423585
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Convolutional Neural Networks (CNNs) often exploit spurious correlations in datasets, learning superficially predictive yet causally irrelevant features, leading to poor generalization and fairness issues. Deep Feature Reweighting (DFR) is a post-hoc technique that reduces a trained model's reliance on spurious correlations by retraining its classification head on a target dataset. However, we show that DFR is fundamentally constrained by operating on entangled features, limiting its ability to amplify the core features while simultaneously suppressing the spurious ones. We trace this entanglement to the ubiquitous Global Average Pooling (GAP) layer, which indiscriminately collapses spatially distinct core and spurious features into a single representation. To address this, we propose Deep Attention Reweighting (DAR), a post-hoc attention-based aggregation module that replaces GAP and is retrained jointly with the classification head. DAR computes an adaptive weighting of spatial locations across feature maps, enabling selective suppression of spurious features before the collapse into entangled features. Across various datasets, metrics, and ablations, DAR consistently outperforms DFR, demonstrating that our attention-based aggregation mitigates GAP-induced entanglement and reduces spurious reliance.

Related papers

• Making Every Step Count: Spatio-Temporal Information Allocation for Imaging Inverse Problems [63.904793414062816]
  We introduce two complementary and training-free components, i.e., Spectrum-based Scheduling (SAS) and Measurement-Prioritized Attention (MPA)<n>SAS distributes the available NFEs over flow time according to the spectrum and logSNR geometry, thus better balancing semantic exploration and detail refinement.<n>MPA guides information toward weakly constrained regions, thereby enhancing semantic and structural fidelity.
  arXiv  Detail & Related papers  (2026-08-12T07:39:22Z)
• Channel-Level Relation to Attentive Aggregation with Neighborhood-Homogeneity Constraint for Point Cloud Analysis [14.63838637349137]
  In 3D point cloud understanding, the core challenge lies in accurately capturing discriminative features within complex neighborhoods.<n>Existing methods explore feature correlation discrimination but are limited to point-level spatial distribution or channel responses.<n>We propose PointCRA, a novel network with a channel-level metric-based enhancement mechanism.
  arXiv  Detail & Related papers  (2026-05-04T08:59:36Z)
• HSFM: Hard-Set-Guided Feature-Space Meta-Learning for Robust Classification under Spurious Correlations [25.503211313768062]
  We propose a bilevel meta-learning method that performs augmentation directly in feature space to improve spurious correlation handling in the classifier head.<n>By operating at the backbone output rather than in pixel space, the method is highly efficient and stable, requiring only a few minutes of training on a single GPU.
  arXiv  Detail & Related papers  (2026-03-31T06:32:56Z)
• i-IF-Learn: Iterative Feature Selection and Unsupervised Learning for High-Dimensional Complex Data [5.761834753104483]
  i-IF-Learn is an unsupervised framework that jointly performs feature selection and clustering.<n>Our core innovation is an adaptive feature selection statistic that effectively combines pseudo-label supervision with unsupervised signals.<n>Experiments on gene microarray and single-cell RNA-seq datasets show that i-IF-Learn significantly surpasses classical and deep clustering baselines.
  arXiv  Detail & Related papers  (2026-03-25T07:35:38Z)
• ResAD++: Towards Class Agnostic Anomaly Detection via Residual Feature Learning [52.11294707895649]
  This paper explores the problem of class-agnostic anomaly detection (AD)<n>The objective is to train one class-agnostic AD model that can generalize to detect anomalies in diverse new classes from different domains without any retraining or fine-tuning on the target data.<n> Comprehensive experiments on eight real-world AD datasets demonstrate that our ResAD++ can achieve remarkable AD results when directly used in new classes.
  arXiv  Detail & Related papers  (2025-09-28T08:41:05Z)
• Minimizing Chebyshev Prototype Risk Magically Mitigates the Perils of Overfitting [1.6574413179773757]
  We develop multicomponent loss functions that reduce intra-class feature correlation and maximize inter-class feature distance. We implement the terms of the Chebyshev Prototype Risk (CPR) bound into our Explicit CPR loss function. Our training algorithm reduces overfitting and improves upon previous approaches in many settings.
  arXiv  Detail & Related papers  (2024-04-10T15:16:04Z)
• CPR++: Object Localization via Single Coarse Point Supervision [55.8671776333499]
  coarse point refinement (CPR) is first attempt to alleviate semantic variance from an algorithmic perspective. CPR reduces semantic variance by selecting a semantic centre point in a neighbourhood region to replace the initial annotated point. CPR++ can obtain scale information and further reduce the semantic variance in a global region.
  arXiv  Detail & Related papers  (2024-01-30T17:38:48Z)
• Self-supervised debiasing using low rank regularization [59.84695042540525]
  Spurious correlations can cause strong biases in deep neural networks, impairing generalization ability. We propose a self-supervised debiasing framework potentially compatible with unlabeled samples. Remarkably, the proposed debiasing framework significantly improves the generalization performance of self-supervised learning baselines.
  arXiv  Detail & Related papers  (2022-10-11T08:26:19Z)
• Fuzzy Attention Neural Network to Tackle Discontinuity in Airway Segmentation [67.19443246236048]
  Airway segmentation is crucial for the examination, diagnosis, and prognosis of lung diseases. Some small-sized airway branches (e.g., bronchus and terminaloles) significantly aggravate the difficulty of automatic segmentation. This paper presents an efficient method for airway segmentation, comprising a novel fuzzy attention neural network and a comprehensive loss function.
  arXiv  Detail & Related papers  (2022-09-05T16:38:13Z)
• Augmented Data as an Auxiliary Plug-in Towards Categorization of Crowdsourced Heritage Data [2.609784101826762]
  We propose a strategy to mitigate the problem of inefficient clustering performance by introducing data augmentation as an auxiliary plug-in. We train a variant of Convolutional Autoencoder (CAE) with augmented data to construct the initial feature space as a novel model for deep clustering.
  arXiv  Detail & Related papers  (2021-07-08T14:09:39Z)
• Adversarial Feature Augmentation and Normalization for Visual Recognition [109.6834687220478]
  Recent advances in computer vision take advantage of adversarial data augmentation to ameliorate the generalization ability of classification models. Here, we present an effective and efficient alternative that advocates adversarial augmentation on intermediate feature embeddings. We validate the proposed approach across diverse visual recognition tasks with representative backbone networks.
  arXiv  Detail & Related papers  (2021-03-22T20:36:34Z)
• High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification [84.43394420267794]
  We propose a novel framework by learning high-order relation and topology information for discriminative features and robust alignment. Our framework significantly outperforms state-of-the-art by6.5%mAP scores on Occluded-Duke dataset.
  arXiv  Detail & Related papers  (2020-03-18T12:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.