Information as Structural Alignment: A Dynamical Theory of Continual Learning

• URL: http://arxiv.org/abs/2604.07108v1
• Date: Wed, 08 Apr 2026 14:00:14 GMT
• Title: Information as Structural Alignment: A Dynamical Theory of Continual Learning
• Abstract summary: This paper introduces the Informational Buildup Framework (IBF), an alternative substrate for continual learning.<n>IBF is based on the premise that information is the achievement of structural alignment rather than stored content.<n>We demonstrate the full lifecycle in a transparent two-dimensional toy model, then validate across three domains.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Catastrophic forgetting is not an engineering failure. It is a mathematical consequence of storing knowledge as global parameter superposition. Existing methods, such as regularization, replay, and frozen subnetworks, add external mechanisms to a shared-parameter substrate. None derives retention from the learning dynamics themselves. This paper introduces the Informational Buildup Framework (IBF), an alternative substrate for continual learning, based on the premise that information is the achievement of structural alignment rather than stored content. In IBF, two equations govern the dynamics: a Law of Motion that drives configuration toward higher coherence, and Modification Dynamics that persistently deform the coherence landscape in response to localized discrepancies. Memory, agency, and self-correction arise from these dynamics rather than being added as separate modules. We first demonstrate the full lifecycle in a transparent two-dimensional toy model, then validate across three domains: a controlled non-stationary world, chess evaluated independently by Stockfish, and Split-CIFAR-100 with a frozen ViT encoder. Across all three, IBF achieves replay-superior retention without storing raw data. We observe near-zero forgetting on CIFAR-100 (BT = -0.004), positive backward transfer in chess (+38.5 cp), and 43% less forgetting than replay in the controlled domain. In chess, the framework achieves a mean behavioral advantage of +88.9 +/- 2.8 cp under independent evaluation, exceeding MLP and replay baselines.

Related papers

• MetaRSI / RSI2: A Meta-Recursive Self-Improving System for Recursive Self-Improving Systems Themselves [65.60481414910728]
  Recursive self-improvement lets a system improve the model-building machinery from its own failures, so every later model inherits the gain.<n>Yet RSI has been validated almost exclusively on coding and formal benchmarks such as science QA and mathematics.<n>We argue RSI must next operate across real, diverse scientific, engineering, and meta-scientific domains, not where formal evaluation is merely tractable.<n>We present MetaRSI-v1, where improvement is the scheduled composition of three typed operators over one unified paradigm.
  arXiv  Detail & Related papers  (2026-09-06T05:15:31Z)
• CaLiSym: Learning Symplectic Dynamics of Real-World Systems through Structured Canonical Lifts [5.265601895449634]
  CaLiSym is a lightweight framework that extends symplectic learning to such systems by changing where the geometric prior is imposed.<n>We instantiate the framework with SympNet predictors and introduce GRB-SympNet, a B-spline variant combining approximation with exact symplectic structure.<n>The lifted dynamics preserve the symplectic form to numerical precision, extending symplectic learning beyond conservative mechanics toward real-world robotics.
  arXiv  Detail & Related papers  (2026-07-07T21:41:59Z)
• Why SWAVE May Not Be All You Need:A Concept-Evolution Retrospective on Complex-Valued Recurrent Language Models [0.0]
  SWave is a complex-valued recurrent language model (169.26M parameters, D=384, L=16, T48) trained on FineWeb-Edu using 2xH100 NVL.<n>It was designed around three founding premises: that representing language as complex waves rather than real-valued numbers enables richer information encoding.
  arXiv  Detail & Related papers  (2026-06-16T16:05:28Z)
• Echo-Memory: A Controlled Study of Memory in Action World Models [58.77172260403133]
  We present a controlled study of memory mechanisms in action-conditioned world models.<n>Existing memory designs are hard to compare because gains are entangled with backbone, training, retrieval, and evaluation differences.<n>We use Echo-Memory to fix the action-to-video interface and varies only how history is stored and read by the generator.
  arXiv  Detail & Related papers  (2026-06-08T17:54:10Z)
• Hierarchical Dual-Subspace Decoupling for Continual Learning in Vision-Language Models [63.169731309799324]
  Class-incremental learning aims to continuously acquire new knowledge while preserving previously learned information, thereby mitigating catastrophic forgetting.<n>We propose HDSD, a Hierarchical Dual-Subspace Decoupling framework for continual learning in vision-language models.<n> Specifically, we introduce a lightweight Feature Modulation Module (FMM) that explicitly decomposes the parameter space into general and task-specific subspaces.
  arXiv  Detail & Related papers  (2026-05-08T09:42:05Z)
• Distilling Deep Reinforcement Learning into Interpretable Fuzzy Rules: An Explainable AI Framework [1.9180507495574395]
  Deep Reinforcement Learning (DRL) agents achieve remarkable performance in continuous control but remain opaque, hindering deployment in safety-critical domains.<n>Existing explainability methods either provide only local insights (SHAP, LIME) or employ oversimplified surrogates failing to capture continuous dynamics.<n>This work proposes a Fuzzy System (FCS) distilling neural policies into human-readable IF-THEN rules through K-Means clustering for state-readable partitioning.<n>Three quantifiable metrics are introduced: Rule Density (FRAD) measuring explanation focus, Fuzzy Set Coverage (FSC) validating vocabulary completeness, and Action Space Granularity (
  arXiv  Detail & Related papers  (2026-02-24T23:53:01Z)
• Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Space [56.37266873329401]
  Large Language Models (LLMs) apply uniform computation to all tokens, despite language exhibiting highly non-uniform information density.<n>We propose $textbfDynamic Large Concept Models (DLCM)$, a hierarchical language modeling framework that learns semantic boundaries from latent representations and shifts from tokens to a compressed concept space where reasoning is more efficient.
  arXiv  Detail & Related papers  (2025-12-31T04:19:33Z)
• Distillation-Guided Structural Transfer for Continual Learning Beyond Sparse Distributed Memory [29.87863386264415]
  We propose a structurally guided continual learning framework that treats distillation not as a regularizer but as a topology-aligned information conduit.<n>SSD identifies neurons with high activation frequency and selectively distills knowledge within previous Top-Kworks and output logits, without requiring replay or task labels.
  arXiv  Detail & Related papers  (2025-12-17T10:17:01Z)
• DynaPURLS: Dynamic Refinement of Part-aware Representations for Skeleton-based Zero-Shot Action Recognition [51.80782323686666]
  We introduce textbfDynaPURLS, a unified framework that establishes robust, multi-scale visual-semantic correspondences.<n>Our framework leverages a large language model to generate hierarchical textual descriptions that encompass both global movements and local body-part dynamics.<n>Experiments on three large-scale benchmark datasets, including NTU RGB+D 60/120 and PKU-MMD, demonstrate that DynaPURLS significantly outperforms prior art.
  arXiv  Detail & Related papers  (2025-12-12T10:39:10Z)
• ExplicitLM: Decoupling Knowledge from Parameters via Explicit Memory Banks [4.099810580680816]
  Large language models suffer from knowledge staleness and lack of interpretability due to implicit knowledge storage.<n>We propose ExplicitLM, a novel architecture featuring a million-scale external memory bank storing human-readable knowledge as token sequences.
  arXiv  Detail & Related papers  (2025-11-03T13:53:19Z)
• Breaking the MoE LLM Trilemma: Dynamic Expert Clustering with Structured Compression [14.086434595924716]
  Mixture-of-Experts (MoE) Large Language Models (LLMs) face a trilemma of load imbalance, parameter redundancy, and communication overhead.<n>We introduce a unified framework based on dynamic expert clustering and structured compression to address these issues cohesively.
  arXiv  Detail & Related papers  (2025-09-27T10:45:58Z)
• Beyond 'Aha!': Toward Systematic Meta-Abilities Alignment in Large Reasoning Models [86.88657425848547]
  Large reasoning models (LRMs) already possess a latent capacity for long chain-of-thought reasoning.<n>We explicitly align models with three meta-abilities: deduction, induction, and abduction, using automatically generated, self-verifiable tasks.<n>Our three stage-pipeline individual alignment, parameter-space merging, and domain-specific reinforcement learning, boosts performance by over 10% relative to instruction-tuned baselines.
  arXiv  Detail & Related papers  (2025-05-15T17:58:33Z)
• Stand-Alone Inter-Frame Attention in Video Models [164.06137994796487]
  We present a new recipe of inter-frame attention block, namely Stand-alone Inter-temporal Attention (SIFA) SIFA remoulds the deformable design via re-scaling the offset predictions by the difference between two frames. We further plug SIFA block into ConvNets and Vision Transformer, respectively, to devise SIFA-Net and SIFA-Transformer.
  arXiv  Detail & Related papers  (2022-06-14T15:51:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.