Shortcut Invariance: Targeted Jacobian Regularization in Disentangled Latent Space

• URL: http://arxiv.org/abs/2511.19525v1
• Date: Mon, 24 Nov 2025 07:09:08 GMT
• Title: Shortcut Invariance: Targeted Jacobian Regularization in Disentangled Latent Space
• Authors: Shivam Pal, Sakshi Varshney, Piyush Rai,
• Abstract summary: Deep neural networks are prone to learning shortcuts, spurious and easily learned correlations.<n>We present a simple and effective training method that renders the classifier functionally invariant to shortcut signals.<n>We analyze this as targeted Jacobian regularization, which forces the classifier to ignore spurious features and rely on more complex, core semantic signals.
• Score: 7.8904984750896885
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep neural networks are prone to learning shortcuts, spurious and easily learned correlations in training data that cause severe failures in out-of-distribution (OOD) generalization. A dominant line of work seeks robustness by learning a robust representation, often explicitly partitioning the latent space into core and spurious components; this approach can be complex, brittle, and difficult to scale. We take a different approach, instead of a robust representation, we learn a robust function. We present a simple and effective training method that renders the classifier functionally invariant to shortcut signals. Our method operates within a disentangled latent space, which is essential as it isolates spurious and core features into distinct dimensions. This separation enables the identification of candidate shortcut features by their strong correlation with the label, used as a proxy for semantic simplicity. The classifier is then desensitized to these features by injecting targeted, anisotropic latent noise during training. We analyze this as targeted Jacobian regularization, which forces the classifier to ignore spurious features and rely on more complex, core semantic signals. The result is state-of-the-art OOD performance on established shortcut learning benchmarks.

Related papers

• Generative Classifiers Avoid Shortcut Solutions [84.23247217037134]
  Discriminative approaches to classification often learn shortcuts that hold in-distribution but fail under minor distribution shift.<n>We show that generative classifiers can avoid this issue by modeling all features, both core and spurious, instead of mainly spurious ones.<n>We find that diffusion-based and autorerimigressive generative classifiers achieve state-of-the-art performance on five standard image and text distribution shift benchmarks.
  arXiv  Detail & Related papers  (2025-12-31T18:31:46Z)
• Semantic Concentration for Self-Supervised Dense Representations Learning [103.10708947415092]
  Image-level self-supervised learning (SSL) has made significant progress, yet learning dense representations for patches remains challenging.<n>This work reveals that image-level SSL avoids over-dispersion by involving implicit semantic concentration.
  arXiv  Detail & Related papers  (2025-09-11T13:12:10Z)
• Spectral regularization for adversarially-robust representation learning [32.84188052937496]
  We propose a new spectral regularizer for representation learning that encourages black-box adversarial robustness in downstream classification tasks. We show that this method is more effective in boosting test accuracy and robustness than previously-proposed methods that regularize all layers of the network.
  arXiv  Detail & Related papers  (2024-05-27T14:01:42Z)
• Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach [102.0769560460338]
  We develop a simple logits approach (LORT) without the requirement of prior knowledge of the number of samples per class. Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
  arXiv  Detail & Related papers  (2024-03-01T03:27:08Z)
• Beyond Prototypes: Semantic Anchor Regularization for Better Representation Learning [82.29761875805369]
  One of the ultimate goals of representation learning is to achieve compactness within a class and well-separability between classes. We propose a novel perspective to use pre-defined class anchors serving as feature centroid to unidirectionally guide feature learning. The proposed Semantic Anchor Regularization (SAR) can be used in a plug-and-play manner in the existing models.
  arXiv  Detail & Related papers  (2023-12-19T05:52:38Z)
• Learning Context-aware Classifier for Semantic Segmentation [88.88198210948426]
  In this paper, contextual hints are exploited via learning a context-aware classifier. Our method is model-agnostic and can be easily applied to generic segmentation models. With only negligible additional parameters and +2% inference time, decent performance gain has been achieved on both small and large models.
  arXiv  Detail & Related papers  (2023-03-21T07:00:35Z)
• Chroma-VAE: Mitigating Shortcut Learning with Generative Classifiers [44.97660597940641]
  We show that generative models alone are not sufficient to prevent shortcut learning. In particular, we propose Chroma-VAE, a two-pronged approach where a VAE is initially trained to isolate the shortcut in a small latent subspace. In addition to demonstrating the efficacy of Chroma-VAE on benchmark and real-world shortcut learning tasks, our work highlights the potential for manipulating the latent space of generative classifiers to isolate or interpret specific correlations.
  arXiv  Detail & Related papers  (2022-11-28T11:27:50Z)
• Regularizing Neural Network Training via Identity-wise Discriminative Feature Suppression [20.89979858757123]
  When the number of training samples is small, or the class labels are noisy, networks tend to memorize patterns specific to individual instances to minimize the training error. This paper explores a remedy by suppressing the network's tendency to rely on instance-specific patterns for empirical error minimisation.
  arXiv  Detail & Related papers  (2022-09-29T05:14:56Z)
• Learning Debiased and Disentangled Representations for Semantic Segmentation [52.35766945827972]
  We propose a model-agnostic and training scheme for semantic segmentation. By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes. Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-10-31T16:15:09Z)
• Beyond cross-entropy: learning highly separable feature distributions for robust and accurate classification [22.806324361016863]
  We propose a novel approach for training deep robust multiclass classifiers that provides adversarial robustness. We show that the regularization of the latent space based on our approach yields excellent classification accuracy.
  arXiv  Detail & Related papers  (2020-10-29T11:15:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.