The Stable Recovery Manifold: Geometric Principles Governing Recoverability in Continual Learning

• URL: http://arxiv.org/abs/2606.13637v1
• Date: Thu, 11 Jun 2026 17:45:41 GMT
• Title: The Stable Recovery Manifold: Geometric Principles Governing Recoverability in Continual Learning
• Abstract summary: Catastrophic forgetting is often viewed as the destruction of previously learned knowledge during sequential learning.<n>We investigate the geometric structure of recoverability in continual learning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Catastrophic forgetting is often viewed as the destruction of previously learned knowledge during sequential learning. Building on the Accessibility Collapse framework, we investigate the geometric structure of recoverability in continual learning. Using Split CIFAR-100 and a sequentially trained ResNet-18, we analyze recoverability, representational drift, and recovery complexity across ten tasks. We introduce Recovery Subspace Dimensionality (k_t), a measure of the minimum number of singular directions required to preserve 90 percent of full probe performance. Contrary to our Recoverability Diffusion hypothesis, recovery dimensionality remains stable throughout training (mean k_t = 8.0) despite substantial representational drift. Principal-angle drift strongly predicts recoverability (r = -0.862), and a simple geometric model explains 82.2 percent of recoverability variance. These findings support the Stable Recovery Manifold hypothesis, suggesting that forgotten knowledge remains compactly decodable despite representational reorganization. The results indicate that catastrophic forgetting is primarily an accessibility and manifold-alignment problem rather than information destruction.

Related papers

• Source-Free Class Relearning: Diagnosing Forgetting in Class Unlearning [22.455121713091245]
  Class unlearning aims to remove a model's ability to recognize designated forget classes while preserving performance on retain classes.<n>We study class relearning in a strictly source-free setting, asking whether a forget class can be recovered using only the unlearned model.
  arXiv  Detail & Related papers  (2026-09-02T02:45:56Z)
• EvoUndo: Recoverability-Constrained Self-Evolution for LLM Agent Harnesses [0.0]
  We introduce EvoUndo, a framework for representing, synthesizing, diagnosing, and independently verifying recoverability of model-generated self-modifications.<n>Across 600 unseen one-shot self-evolution tasks, we identify 197 capability-improving mutations that fail recoverability verification.
  arXiv  Detail & Related papers  (2026-08-28T14:15:36Z)
• Recover, Discover, Plan: Learning Skills and Concepts from Robot Failures [50.668639270656115]
  ReSYNC is a robot that learns skills and concepts and refines its abstraction library to support abstract planning.<n>We show that ReSYNC's ability to continually expand and refine its abstraction library allows it to solve long-horizon, previously unseen problems.<n>Overall, ReSYNC represents a significant step toward robots that autonomously acquire abstractions for scalable, failure-aware planning.
  arXiv  Detail & Related papers  (2026-06-16T17:59:57Z)
• Catastrophic Forgetting as Accessibility Collapse: A Three-Level Framework for Knowledge Persistence in Continual Learning [0.0]
  Catastrophic forgetting is commonly interpreted as the irreversible erasure of previously acquired knowledge during sequential learning.<n>In this work, we investigate an alternative perspective: that forgetting may arise not from complete destruction of task representations but from a loss of accessibility to preserved information.
  arXiv  Detail & Related papers  (2026-06-04T11:25:33Z)
• How Neural Losses Shape VAE Latents [15.636976191219448]
  We show that the choice of reconstruction loss reshapes the rate-distortion problem itself.<n>We prove and verify that augmenting pointwise reconstruction with neural terms, such as perceptual and adversarial objectives, reduces the amount of information stored in the latent representations.
  arXiv  Detail & Related papers  (2026-05-30T09:20:56Z)
• Next-Acceleration-Scale Prediction for Autoregressive MRI Reconstruction [52.32112533846212]
  MRI reconstruction is an inherently ill-posed inverse problem, since incomplete measurements admit many plausible solutions.<n>We address this limitation by moving reconstruction to discrete multi-scale latent space and posing it as autoregressive next-acceleration-scale prediction.<n>We show that our approach delivers improved reconstruction performance across diverse sampling patterns under extreme undersampling.
  arXiv  Detail & Related papers  (2026-05-19T04:40:50Z)
• Attribution-Based Neuron Utility for Plasticity Restoration in Deep Networks [0.0]
  Continual learning research attempts to conserve two fundamental capabilities: new knowledge acquisition and the preservation of previously acquired knowledge.<n>Recent research has identified multiple mechanisms underlying this phenomenon, including neuron saturation, norm growth, and loss of useful curvature directions.<n> Adaptive reset-based interventions have emerged as practical solutions to restore trainability.
  arXiv  Detail & Related papers  (2026-05-07T18:42:22Z)
• ARES: Scalable and Practical Gradient Inversion Attack in Federated Learning through Activation Recovery [73.8181449261685]
  Federated Learning (FL) enables collaborative model training by sharing model updates instead of raw data, aiming to protect user privacy.<n>Recent studies reveal that these shared updates can inadvertently leak sensitive training data through gradient inversion attacks (GIAs)
  arXiv  Detail & Related papers  (2026-03-18T11:40:44Z)
• Embedding Classical Balance Control Principles in Reinforcement Learning for Humanoid Recovery [1.0883174135300417]
  Humans are vulnerable to falls and unrecoverable failure states, limiting their practical utility in unstructured environments.<n>We present a unified RL policy that addresses this limitation by embedding classical balance metrics.<n>The policy achieves a 93.4% recovery rate across randomized initial poses and unscripted fall configurations.
  arXiv  Detail & Related papers  (2026-03-09T17:02:30Z)
• Beyond Progress Measures: Theoretical Insights into the Mechanism of Grokking [50.465604300990904]
  Grokking refers to the abrupt improvement in test accuracy after extended overfitting.<n>In this work, we investigate the grokking mechanism underlying the Transformer in the task of prime number operations.
  arXiv  Detail & Related papers  (2025-04-04T04:42:38Z)
• Zero-Shot Self-Consistency Learning for Seismic Irregular Spatial Sampling Reconstruction [6.313946204460284]
  We propose a zero-shot self-consistency learning strategy and employ an extremely lightweight network for seismic data reconstruction. Our method does not require additional datasets and utilizes the correlations among different parts of the data to design a self-consistency learning loss function.
  arXiv  Detail & Related papers  (2024-11-01T11:59:28Z)
• On the Dynamics Under the Unhinged Loss and Beyond [104.49565602940699]
  We introduce the unhinged loss, a concise loss function, that offers more mathematical opportunities to analyze closed-form dynamics. The unhinged loss allows for considering more practical techniques, such as time-vary learning rates and feature normalization.
  arXiv  Detail & Related papers  (2023-12-13T02:11:07Z)
• Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillation [110.61853418925219]
  We build a stronger version of the dataset reconstruction attack and show how it can provably recover the emphentire training set in the infinite width regime. We show that both theoretically and empirically, reconstructed images tend to "outliers" in the dataset. These reconstruction attacks can be used for textitdataset distillation, that is, we can retrain on reconstructed images and obtain high predictive accuracy.
  arXiv  Detail & Related papers  (2023-02-02T21:41:59Z)
• Grading Loss: A Fracture Grade-based Metric Loss for Vertebral Fracture Detection [58.984536305767996]
  We propose a representation learning-inspired approach for automated vertebral fracture detection. We present a novel Grading Loss for learning representations that respect Genant's fracture grading scheme. On a publicly available spine dataset, the proposed loss function achieves a fracture detection F1 score of 81.5%.
  arXiv  Detail & Related papers  (2020-08-18T10:03:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.