Non-Equilibrium Stochastic Dynamics as a Unified Framework for Insight and Repetitive Learning: A Kramers Escape Approach to Continual Learning

• URL: http://arxiv.org/abs/2604.04154v1
• Date: Sun, 05 Apr 2026 15:42:23 GMT
• Title: Non-Equilibrium Stochastic Dynamics as a Unified Framework for Insight and Repetitive Learning: A Kramers Escape Approach to Continual Learning
• Abstract summary: Continual learning in artificial neural networks is fundamentally limited by the stability-plasticity dilemma.<n>We show that both problems admit a common resolution within non-equilibrium statistical physics.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continual learning in artificial neural networks is fundamentally limited by the stability--plasticity dilemma: systems that retain prior knowledge tend to resist acquiring new knowledge, and vice versa. Existing approaches, most notably elastic weight consolidation~(EWC), address this empirically without a physical account of why plasticity eventually collapses as tasks accumulate. Separately, the distinction between sudden insight and gradual skill acquisition through repetitive practice has lacked a unified theoretical description. Here, we show that both problems admit a common resolution within non-equilibrium statistical physics. We model the state of a learning system as a particle evolving under Langevin dynamics on a double-well energy landscape, with the noise amplitude governed by a time-dependent effective temperature $T(t)$. The probability density obeys a Fokker--Planck equation, and transitions between metastable states are governed by the Kramers escape rate $k = (ω_0ω_b/2π)\,e^{-ΔE/T}$. We make two contributions. First, we identify the EWC penalty term as an energy barrier whose height grows linearly with the number of accumulated tasks, yielding an exponential collapse of the transition rate predicted analytically and confirmed numerically. Second, we show that insight and repetitive learning correspond to two qualitatively distinct temperature protocols within the same Fokker--Planck equation: insight events produce transient spikes in $T(t)$ that drive rapid barrier crossing, whereas repetitive practice operates at a modestly elevated but fixed temperature, achieving transitions through sustained stochastic diffusion. These results establish a physically grounded framework for understanding plasticity and its failure in continual learning systems, and suggest principled design criteria for adaptive noise schedules in artificial intelligence.

Related papers

• When does dissipation help neural surrogates learn open quantum dynamics? [0.0]
  We show that dissipation can enhance the learnability of open quantum dynamics.<n>But fidelity alone is insufficient to distinguish genuine dynamical learning from steady-state trivialization.
  arXiv  Detail & Related papers  (2026-06-22T19:47:40Z)
• Gradient descent at the Edge of Stability: free energy model and kinetic description of the two-layer network [0.0]
  We study the dynamics of gradient descent in the Edge of Stability regime.<n>We propose a continuous-time effective model that tracks the evolution of the average trajectory.<n>We show that this equation can be interpreted as a Wasserstein-2 gradient flow of a macroscopic free energy.
  arXiv  Detail & Related papers  (2026-06-03T18:11:33Z)
• Analyzing and Improving Diffusion Models for Time-Series Data Imputation: A Proximal Recursion Perspective [45.713195454899875]
  Diffusion models (DMs) have shown promise for Time-Series Data Imputation.<n>DMs' performance remains inconsistent in complex scenarios.<n>We propose a novel framework called SPIRIT (Semi-Proximal Transport Regularized time-series Imputation)
  arXiv  Detail & Related papers  (2026-02-01T12:11:57Z)
• Transformer Learning of Chaotic Collective Dynamics in Many-Body Systems [0.0]
  We show that a self-attention-based transformer framework provides an effective approach for modeling chaotic collective dynamics.<n>We study the one-dimensional semiclassical Holstein model, where interaction quenches induce strongly nonlinear and chaotic dynamics.<n>Our results establish self-attention as a powerful mechanism for learning effective reduced dynamics in chaotic many-body systems.
  arXiv  Detail & Related papers  (2026-01-27T01:33:33Z)
• Identifiable learning of dissipative dynamics [25.409059056398124]
  We introduce I-OnsagerNet, a neural framework that learns dissipative dynamics directly from trajectories.<n>I-OnsagerNet extends the Onsager principle to guarantee that the learned potential is obtained from the stationary density.<n>Our approach enables us to calculate the entropy production and to quantify irreversibility, offering a principled way to detect and quantify deviations from equilibrium.
  arXiv  Detail & Related papers  (2025-10-28T07:57:14Z)
• Emergence of Superposition: Unveiling the Training Dynamics of Chain of Continuous Thought [64.43689151961054]
  We theoretically analyze the training dynamics of a simplified two-layer transformer on the directed graph reachability problem.<n>Our analysis reveals that during training using continuous thought, the index-matching logit will first increase and then remain bounded under mild assumptions.
  arXiv  Detail & Related papers  (2025-09-27T15:23:46Z)
• Data-driven particle dynamics: Structure-preserving coarse-graining for emergent behavior in non-equilibrium systems [0.8796261172196743]
  Multiscale systems are notoriously challenging to simulate as shorttemporal scales must be appropriately linked to emergent bulk physics.<n>We propose a framework using the metriplectic bracket formalism that preserves discrete notions of the first and second laws of thermodynamics.<n>We provide open-source implementations in both PyTorch and LAMMPS, enabling large-scale inference and rearrangement to diverse particle-based systems.
  arXiv  Detail & Related papers  (2025-08-18T02:10:18Z)
• Noradrenergic-inspired gain modulation attenuates the stability gap in joint training [44.99833362998488]
  Studies in continual learning have identified a transient drop in performance on mastered tasks when assimilating new ones, known as the stability gap.<n>We argue that it reflects an imbalance between rapid adaptation and robust retention at task boundaries.<n>Inspired by locus coeruleus mediated noradrenergic bursts, we propose uncertainty-modulated gain dynamics.
  arXiv  Detail & Related papers  (2025-07-18T16:34:06Z)
• Unusual Diffusivity in Strongly Disordered Quantum Lattices: Random Dimer Model [0.0]
  We study highly disordered quantum lattices using superconducting qubits. Our predictions challenge conventional understanding of incoherent hopping under strong disorder. This offers new insights to optimize disordered systems for optoelectrical and quantum information technologies.
  arXiv  Detail & Related papers  (2024-05-31T14:09:48Z)
• Real-time dynamics of false vacuum decay [49.1574468325115]
  We investigate false vacuum decay of a relativistic scalar field in the metastable minimum of an asymmetric double-well potential. We employ the non-perturbative framework of the two-particle irreducible (2PI) quantum effective action at next-to-leading order in a large-N expansion.
  arXiv  Detail & Related papers  (2023-10-06T12:44:48Z)
• Machine learning in and out of equilibrium [58.88325379746631]
  Our study uses a Fokker-Planck approach, adapted from statistical physics, to explore these parallels. We focus in particular on the stationary state of the system in the long-time limit, which in conventional SGD is out of equilibrium. We propose a new variation of Langevin dynamics (SGLD) that harnesses without replacement minibatching.
  arXiv  Detail & Related papers  (2023-06-06T09:12:49Z)
• The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion [29.489737359897312]
  We study the limiting dynamics of deep neural networks trained with gradient descent (SGD) We show that the key ingredient driving these dynamics is not the original training loss, but rather the combination of a modified loss, which implicitly regularizes the velocity and probability currents, which cause oscillations in phase space.
  arXiv  Detail & Related papers  (2021-07-19T20:18:57Z)
• Training Generative Adversarial Networks by Solving Ordinary Differential Equations [54.23691425062034]
  We study the continuous-time dynamics induced by GAN training. From this perspective, we hypothesise that instabilities in training GANs arise from the integration error. We experimentally verify that well-known ODE solvers (such as Runge-Kutta) can stabilise training.
  arXiv  Detail & Related papers  (2020-10-28T15:23:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.