Diagnosing Shortcut-Induced Rigidity in Continual Learning: The Einstellung Rigidity Index (ERI)

• URL: http://arxiv.org/abs/2510.00475v1
• Date: Wed, 01 Oct 2025 03:52:40 GMT
• Title: Diagnosing Shortcut-Induced Rigidity in Continual Learning: The Einstellung Rigidity Index (ERI)
• Abstract summary: Shortcut features undermine robustness and reduce reliability under distribution shifts.<n>In continual learning (CL), the consequences of shortcut exploitation can persist and intensify.<n>In CL, shortcut-induced rigidity throttles the acquisition of new ones.
• Score: 7.587193411022608
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep neural networks frequently exploit shortcut features, defined as incidental correlations between inputs and labels without causal meaning. Shortcut features undermine robustness and reduce reliability under distribution shifts. In continual learning (CL), the consequences of shortcut exploitation can persist and intensify: weights inherited from earlier tasks bias representation reuse toward whatever features most easily satisfied prior labels, mirroring the cognitive Einstellung effect, a phenomenon where past habits block optimal solutions. Whereas catastrophic forgetting erodes past skills, shortcut-induced rigidity throttles the acquisition of new ones. We introduce the Einstellung Rigidity Index (ERI), a compact diagnostic that disentangles genuine transfer from cue-inflated performance using three interpretable facets: (i) Adaptation Delay (AD), (ii) Performance Deficit (PD), and (iii) Relative Suboptimal Feature Reliance (SFR_rel). On a two-phase CIFAR-100 CL benchmark with a deliberately spurious magenta patch in Phase 2, we evaluate Naive fine-tuning (SGD), online Elastic Weight Consolidation (EWC_on), Dark Experience Replay (DER++), Gradient Projection Memory (GPM), and Deep Generative Replay (DGR). Across these continual learning methods, we observe that CL methods reach accuracy thresholds earlier than a Scratch-T2 baseline (negative AD) but achieve slightly lower final accuracy on patched shortcut classes (positive PD). Masking the patch improves accuracy for CL methods while slightly reducing Scratch-T2, yielding negative SFR_rel. This pattern indicates the patch acted as a distractor for CL models in this setting rather than a helpful shortcut.

Related papers

• Prevalence calibration as shortcut mitigation [1.8620637029128548]
  Shortcut learning denotes the widespread situation in which a classifier exploits spurious correlations rather than diagnostic features.<n>We show that unconstrained learning implicitly calibrates each shortcut group to its training set disease prevalence, rendering the resulting classifier necessarily over-confident in one group and under-confident in the other.
  arXiv  Detail & Related papers  (2026-09-07T19:38:20Z)
• Robust Activation Map Rectification for Weakly Supervised Volumetric Segmentation: Temporal Coherence as a Free Lunch [31.156115062163412]
  Weakly supervised segmentation relies heavily on class activation maps (CAMs) to initially localize target regions.<n>Existing remedies typically introduce additional training stages or prototype learning, increasing computational cost and reducing robustness.<n>We propose a training-free prototype-free framework that rectifies unreliable CAMs by exploiting temporal and structural coherence in volumetric data as a free lunch.
  arXiv  Detail & Related papers  (2026-07-22T08:07:38Z)
• LCPNet: Latent Consistent Proximal Unfolding Network for Infrared Small Target Detection [50.3892176822492]
  Infrared small target detection (IRSTD) aims to identify long distance small targets from complex infrared backgrounds.<n>Deep learning methods have improved IRSTD by learning discriminative image-to-mask mappings.<n>Deep unfolding methods partially address this issue by embedding model-driven iterations into neural networks.<n>We propose Latent Consistent Proximal unfolding network (LCPNet) to address these limitations.
  arXiv  Detail & Related papers  (2026-07-06T02:15:09Z)
• DriftScope: Measuring The Hidden Effects of Diffusion Model Adaptation [50.584834483403675]
  Adapting pre-trained text-to-image diffusion models, whether to learn new visual concepts or erase unwanted ones, is routinely evaluated on its intended effects alone.<n>We demonstrate that adaptation systematically damages semantically unrelated concepts in ways that aggregate metrics structurally cannot surface.<n>We introduce DriftScope, a prompt-level diagnostic tool that takes any two model checkpoints and returns a ranked list of tokens whose visual concepts have shifted most between them.
  arXiv  Detail & Related papers  (2026-06-30T20:57:51Z)
• Where to Refine, When to Stop: Rethinking Redundancy via Latent Discrepancy for Efficient Visual Autoregressive Generation [50.139984798361375]
  We propose a training-free framework that removes redundancy via latent discrepancy.<n> LD-Pruning substantially reduces inference while maintaining high generation quality, achieving up to 2.35x speedup on Infinity-8B.
  arXiv  Detail & Related papers  (2026-05-29T19:34:39Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• Constraint-Rectified Training for Efficient Chain-of-Thought [60.52883907721588]
  Chain-of-Thought (CoT) has significantly enhanced the reasoning capabilities of Large Language Models (LLMs)<n>While longer reasoning traces can improve answer quality and unlock abilities such as self-correction, they also incur high inference costs and often introduce redundant steps, known as overthinking.<n>Recent research seeks to develop efficient reasoning strategies that balance reasoning length and accuracy.
  arXiv  Detail & Related papers  (2026-02-13T02:13:45Z)
• Beyond Sharpness: A Flatness Decomposition Framework for Efficient Continual Learning [27.583428955764774]
  Continual Learning aims to enable models to sequentially learn multiple tasks without forgetting previous knowledge.<n>Existing sharpness-aware methods for Continual Learning suffer from two key limitations.<n>We propose FLAD, a novel optimization framework that decomposes perturbations into sharpness-aligned and gradient-noise components.
  arXiv  Detail & Related papers  (2026-01-12T15:17:04Z)
• Randomized Neural Network with Adaptive Forward Regularization for Online Task-free Class Incremental Learning [16.323995111105884]
  We propose a neural network (NN) with forward regularization (-F) to resist forgetting and enhance learning performance.<n>We derive the algorithm of the ensemble deep random vector functional link network (edRVFL) with adjustable forward regularization (-kF)<n>edRVFL-kF generates one-pass closed-form incremental updates and variable learning rates, effectively avoiding past replay and catastrophic forgetting.<n>We improve it to the plug-and-play edRVFL-kF-Bayes, enabling all hard ks in multiple sub-learners to be self-adaptively determined.
  arXiv  Detail & Related papers  (2025-10-24T11:50:13Z)
• STABLE: Gated Continual Learning for Large Language Models [0.0]
  STABLE is a gated continual self editing framework that constrains forgetting during sequential updates.<n>Each candidate edit is evaluated against a stability budget using one of three metrics.<n>Experiments on the Qwen-2.5-7B model show that gating effectively mitigates forgetting while preserving adaptability.
  arXiv  Detail & Related papers  (2025-10-17T16:14:05Z)
• Forward-Only Continual Learning [8.873948519614244]
  Catastrophic forgetting remains a central challenge in continual learning.<n>We propose FoRo, a forward-only, gradient-free continual learning method.<n>Experiments show that FoRo significantly reduces average forgetting and improves accuracy.
  arXiv  Detail & Related papers  (2025-09-01T15:10:38Z)
• R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning [80.104336426172]
  Chain-of-thought (CoT) enhances problem-solving ability of large language models.<n>CoT incurs substantial inference cost due to long autoregressive trajectories.<n>We introduce R-Stitch, a training-free hybrid decoding framework.
  arXiv  Detail & Related papers  (2025-07-23T08:14:36Z)
• EKPC: Elastic Knowledge Preservation and Compensation for Class-Incremental Learning [53.88000987041739]
  Class-Incremental Learning (CIL) aims to enable AI models to continuously learn from sequentially arriving data of different classes over time.<n>We propose the Elastic Knowledge Preservation and Compensation (EKPC) method, integrating Importance-aware importance Regularization (IPR) and Trainable Semantic Drift Compensation (TSDC) for CIL.
  arXiv  Detail & Related papers  (2025-06-14T05:19:58Z)
• Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning [48.11265601808718]
  We show that standard self-supervised contrastive learning objectives implicitly approximate a supervised variant we call the negatives-only supervised contrastive loss (NSCL)<n>We prove that the gap between the CL and NSCL losses vanishes as the number of semantic classes increases, under a bound that is both label-agnostic and architecture-independent.
  arXiv  Detail & Related papers  (2025-06-04T19:43:36Z)
• Replay-Free Continual Low-Rank Adaptation with Dynamic Memory [62.85596937435928]
  We revisit continual learning, which enables pre-trained vision transformers (ViTs) to sequentially fine-tune on new downstream tasks over time.<n>Recent studies highlight a crossover between CL techniques and parameter-efficient fine-tuning.<n>We propose a novel PEFT-CL method called Dual Low-Rank Adaptation (DualLoRA)
  arXiv  Detail & Related papers  (2024-11-01T14:28:39Z)
• Improving Dictionary Learning with Gated Sparse Autoencoders [8.3037652157611]
  Gated Sparse Autoencoder (Gated SAE) is a technique for unsupervised discovery of interpretable features in language models' (LMs) activations. In SAEs, the L1 penalty used to encourage sparsity introduces many undesirable biases, such as shrinkage. In training SAEs on LMs of up to 7B parameters, Gated SAEs solve shrinkage, and require half as many firing features to achieve comparable reconstruction fidelity.
  arXiv  Detail & Related papers  (2024-04-24T17:47:22Z)
• Soft ascent-descent as a stable and flexible alternative to flooding [6.527016551650139]
  We propose a softened, pointwise mechanism called SoftAD that downweights points on the borderline, limits the effects of outliers, and retains the ascent-descent effect of flooding. We demonstrate how SoftAD can realize classification accuracy competitive with flooding while enjoying a much smaller loss generalization gap and model norm.
  arXiv  Detail & Related papers  (2023-10-16T02:02:56Z)
• DELTA: Dynamic Embedding Learning with Truncated Conscious Attention for CTR Prediction [61.68415731896613]
  Click-Through Rate (CTR) prediction is a pivotal task in product and content recommendation. We propose a model that enables Dynamic Embedding Learning with Truncated Conscious Attention for CTR prediction.
  arXiv  Detail & Related papers  (2023-05-03T12:34:45Z)
• Dense Label Encoding for Boundary Discontinuity Free Rotation Detection [69.75559390700887]
  This paper explores a relatively less-studied methodology based on classification. We propose new techniques to push its frontier in two aspects. Experiments and visual analysis on large-scale public datasets for aerial images show the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-11-19T05:42:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.