Catastrophic Forgetting as Accessibility Collapse: A Three-Level Framework for Knowledge Persistence in Continual Learning

• URL: http://arxiv.org/abs/2606.06032v1
• Date: Thu, 04 Jun 2026 11:25:33 GMT
• Title: Catastrophic Forgetting as Accessibility Collapse: A Three-Level Framework for Knowledge Persistence in Continual Learning
• Abstract summary: Catastrophic forgetting is commonly interpreted as the irreversible erasure of previously acquired knowledge during sequential learning.<n>In this work, we investigate an alternative perspective: that forgetting may arise not from complete destruction of task representations but from a loss of accessibility to preserved information.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Catastrophic forgetting is commonly interpreted as the irreversible erasure of previously acquired knowledge during sequential learning. In this work, we investigate an alternative perspective: that forgetting may arise not from complete destruction of task representations but from a loss of accessibility to preserved information. We introduce a three-level framework separating knowledge storage, representation, and accessibility, and evaluate each component through a series of continual-learning experiments on sequential CIFAR-100 classification using ResNet-18. Our analysis combines checkpoint persistence, linear probing, representation geometry, classifier-reset recovery, and layer-wise recoverability experiments. We observe complete behavioral forgetting of earlier tasks, with task accuracy collapsing from 54.8% to 0%, while linear probe performance retains approximately 76% of the original representational information. Furthermore, retraining only the final classifier restores 75.7% of the original task performance without modifying the backbone network. Layer-wise analysis reveals that early and intermediate layers preserve highly recoverable task information despite severe degradation at later stages. Projection-energy and principal-angle analyses indicate that retained knowledge persists as distributed high-dimensional representations rather than through preservation of a small dominant subspace. These findings suggest that catastrophic forgetting is better characterized as an accessibility failure than complete representational erasure, and that substantial task-relevant information remains embedded within neural representations even after functional forgetting has occurred.

Related papers

• Measure, Don't Optimize: Forecasting Recovery in LLM Unlearning [57.920153607557744]
  J-Access is an inference-time audit that measures how often target concepts remain accessible along the model's output pathway.<n>We audit 398 public unlearned models spanning eight unlearning methods.
  arXiv  Detail & Related papers  (2026-08-11T20:16:20Z)
• The Stable Recovery Manifold: Geometric Principles Governing Recoverability in Continual Learning [0.0]
  Catastrophic forgetting is often viewed as the destruction of previously learned knowledge during sequential learning.<n>We investigate the geometric structure of recoverability in continual learning.
  arXiv  Detail & Related papers  (2026-06-11T17:45:41Z)
• Lost or Hidden? A Concept-Level Forgetting in Supervised Continual Learning [12.190665911412466]
  We propose a framework that decomposes forgetting into apparent concept deletion, recoverability, and decodability.<n>We show that a large portion of seemingly lost concept-level information can often be recovered under linearity assumption.<n>Overall, our findings suggest that a significant part of concept-level forgetting can be attributed to changes in the representational accessibility.
  arXiv  Detail & Related papers  (2026-05-10T20:07:10Z)
• Provable Effects of Data Replay in Continual Learning: A Feature Learning Perspective [28.881077229756404]
  We present a comprehensive theoretical framework for analyzing full data-replay training in continual learning.<n>We identify the signal-to-noise ratio (SNR) as a critical factor affecting forgetting.<n>We uncover a novel insight into task ordering: prioritizing higher-signal tasks not only facilitates learning of lower-signal tasks but also helps prevent catastrophic forgetting.
  arXiv  Detail & Related papers  (2026-02-02T20:21:17Z)
• SINDER: Repairing the Singular Defects of DINOv2 [61.98878352956125]
  Vision Transformer models trained on large-scale datasets often exhibit artifacts in the patch token they extract. We propose a novel fine-tuning smooth regularization that rectifies structural deficiencies using only a small dataset.
  arXiv  Detail & Related papers  (2024-07-23T20:34:23Z)
• Strike a Balance in Continual Panoptic Segmentation [60.26892488010291]
  We introduce past-class backtrace distillation to balance the stability of existing knowledge with the adaptability to new information. We also introduce a class-proportional memory strategy, which aligns the class distribution in the replay sample set with that of the historical training data. We present a new method named Continual Panoptic Balanced (BalConpas)
  arXiv  Detail & Related papers  (2024-07-23T09:58:20Z)
• A Scalable and Generalized Deep Learning Framework for Anomaly Detection in Surveillance Videos [0.47279903800557493]
  Anomaly detection in videos is challenging due to the complexity, noise, and diverse nature of activities such as violence, shoplifting, and vandalism. Existing approaches have struggled to apply deep learning models across different anomaly tasks without extensive retraining. A new DL framework is introduced in this study, consisting of three key components: transfer learning to enhance feature generalization, model fusion to improve feature representation, and multi-task classification. Empirical evaluations demonstrate the framework's effectiveness, achieving an accuracy of 97.99% on the RLVS dataset (violence detection), 83.59% on the UCF dataset (shoplifting detection
  arXiv  Detail & Related papers  (2024-07-17T22:41:12Z)
• On the Dynamics Under the Unhinged Loss and Beyond [104.49565602940699]
  We introduce the unhinged loss, a concise loss function, that offers more mathematical opportunities to analyze closed-form dynamics. The unhinged loss allows for considering more practical techniques, such as time-vary learning rates and feature normalization.
  arXiv  Detail & Related papers  (2023-12-13T02:11:07Z)
• Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality [55.88910947643436]
  Self-supervised pre-training is essential for handling vast quantities of unlabeled data in practice. HiDe-Prompt is an innovative approach that explicitly optimize the hierarchical components with an ensemble of task-specific prompts and statistics. Our experiments demonstrate the superior performance of HiDe-Prompt and its robustness to pre-training paradigms in continual learning.
  arXiv  Detail & Related papers  (2023-10-11T06:51:46Z)
• A Study of Forward-Forward Algorithm for Self-Supervised Learning [65.268245109828]
  We study the performance of forward-forward vs. backpropagation for self-supervised representation learning. Our main finding is that while the forward-forward algorithm performs comparably to backpropagation during (self-supervised) training, the transfer performance is significantly lagging behind in all the studied settings.
  arXiv  Detail & Related papers  (2023-09-21T10:14:53Z)
• Detachedly Learn a Classifier for Class-Incremental Learning [11.865788374587734]
  We present an analysis that the failure of vanilla experience replay (ER) comes from unnecessary re-learning of previous tasks and incompetence to distinguish current task from the previous ones. We propose a novel replay strategy task-aware experience replay. Experimental results show our method outperforms current state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-02-23T01:35:44Z)
• Proper Reuse of Image Classification Features Improves Object Detection [4.240984948137734]
  A common practice in transfer learning is to initialize the downstream model weights by pre-training on a data-abundant upstream task. Recent works show this is not strictly necessary under longer training regimes and provide recipes for training the backbone from scratch. We show that an extreme form of knowledge preservation -- freezing the classifier-d backbone -- consistently improves many different detection models.
  arXiv  Detail & Related papers  (2022-04-01T14:44:47Z)
• Reducing Representation Drift in Online Continual Learning [87.71558506591937]
  We study the online continual learning paradigm, where agents must learn from a changing distribution with constrained memory and compute. In this work we instead focus on the change in representations of previously observed data due to the introduction of previously unobserved class samples in the incoming data stream.
  arXiv  Detail & Related papers  (2021-04-11T15:19:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.