The Forward-Backward Disconnect: State Dynamics, Credit Assignment, and Biological Grounding in Neural Computation

• URL: http://arxiv.org/abs/2608.19995v1
• Date: Thu, 20 Aug 2026 13:09:10 GMT
• Title: The Forward-Backward Disconnect: State Dynamics, Credit Assignment, and Biological Grounding in Neural Computation
• Abstract summary: We develop a taxonomy spanning neural model families along three axes: state-dynamics structure, credit-assignment mechanism, and biological grounding.<n>Across static, recurrent, attention-based, state-space, continuous-time, implicit, spiking, biologically plausible, and neuromorphic families, forward dynamics have diversified.<n>Highest demonstrated scales remain concentrated in global or closely gradient-derived error-propagation mechanisms.
• Score: 0.3695480271934741
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A recurring pattern in neural computation is the reintroduction of dynamical and biological structure into models originally simplified for scalable optimization. Early feedforward networks reduced biological neurons to threshold or rate-like summation units, an abstraction compatible with global-gradient training at scale. Since then, forward computation has diversified: modern architectures carry recurrent state, retrieve from long contexts through attention and associative memory, compress histories through structured state-space dynamics, evolve in continuous time, settle to implicit equilibria, and communicate through sparse spikes. Training has diversified less. Scalable learning remains concentrated around backpropagation, backpropagation through time, adjoint methods, implicit differentiation, and surrogate-gradient variants. We call this asymmetry the forward-backward disconnect and develop a taxonomy spanning neural model families along three coupled axes: state-dynamics structure, credit-assignment mechanism, and biological grounding. Forward and learning grounding are treated separately, and the unit of analysis is the architecture-learning configuration rather than the architecture name alone. Across static, recurrent, attention-based, state-space, continuous-time, implicit, spiking, biologically plausible, and neuromorphic families, forward dynamics have diversified while the highest demonstrated scales remain concentrated in global or closely gradient-derived error-propagation mechanisms. Closing this disconnect requires better alignment among state dynamics, credit assignment, and computational substrate.

Related papers

• Divisive Normalization Shapes Low-Rank Slow Manifolds for Continuous Working Memory [6.1311571881384745]
  We show that divisive normalization is mathematically essential to prevent manifold shattering under time-varying inputs.<n>Our findings identify divisive normalization as a critical computational mechanism for learning high-fidelity continuous representations.
  arXiv  Detail & Related papers  (2026-08-03T09:20:05Z)
• Preserving Plasticity in Continual Learning via Dynamical Isometry [5.9237150961415]
  Continual training of deep neural networks under non-stationarity leads to a progressive loss of plasticity, eventually limiting further learning.<n>We relate plasticity to the empirical Neural Kernel, and identify dynamical isometry as a key mechanism for preserving plasticity in continual learning.
  arXiv  Detail & Related papers  (2026-06-08T17:24:15Z)
• AtomicMotion: Learning Human Motion From Different Human Parts [9.72019653340761]
  We present AtomicMotion, a framework designed to decouple and re-integrate body dynamics through three core innovations.<n>First, we introduce a logical body partitioning scheme that decomposes the skeleton into five distinct clusters based on functional intent.<n>Second, to robustly map sparse inputs to high-dimensional poses, we employ a masked full-body pre-conditioning strategy.<n>Third, addressing the limitations of vanilla spatial attention, we propose Kinematic Attention.
  arXiv  Detail & Related papers  (2026-05-21T15:39:38Z)
• Kirchhoff-Inspired Neural Networks for Evolving High-Order Perception [51.28469959548178]
  We propose the Kirchhoff-Inspired Neural Network (KINN), a state-variable-based network architecture constructed based on Kirchhoff's current law.<n>KINN derives numerically stable state updates from fundamental ordinary differential equations, enabling the explicit decoupling and encoding of higher-order evolutionary components.
  arXiv  Detail & Related papers  (2026-03-25T06:17:31Z)
• Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learning [58.533203990515034]
  Scaling neural networks has driven breakthrough advances in machine learning, yet this paradigm fails in deep reinforcement learning (DRL)<n>We show that dynamic sparse training strategies provide module-specific benefits that complement the primary scalability foundation established by architectural improvements.<n>We finally distill these insights into Module-Specific Training (MST), a practical framework that exploits the benefits of architectural improvements and demonstrates substantial scalability gains across diverse RL algorithms without algorithmic modifications.
  arXiv  Detail & Related papers  (2025-10-14T03:03:08Z)
• Learning by Steering the Neural Dynamics: A Statistical Mechanics Perspective [0.0]
  We study how neural dynamics can support fully local, distributed learning.<n>We propose a biologically plausible algorithm for supervised learning with any binary recurrent network.
  arXiv  Detail & Related papers  (2025-10-13T22:28:34Z)
• Dynamical Learning in Deep Asymmetric Recurrent Neural Networks [1.3421746809394772]
  We show that asymmetric deep recurrent neural networks give rise to an exponentially large, dense accessible manifold of internal representations.<n>We propose a distributed learning scheme in which input-output associations emerge naturally from the recurrent dynamics.
  arXiv  Detail & Related papers  (2025-09-05T12:05:09Z)
• Langevin Flows for Modeling Neural Latent Dynamics [81.81271685018284]
  We introduce LangevinFlow, a sequential Variational Auto-Encoder where the time evolution of latent variables is governed by the underdamped Langevin equation.<n>Our approach incorporates physical priors -- such as inertia, damping, a learned potential function, and forces -- to represent both autonomous and non-autonomous processes in neural systems.<n>Our method outperforms state-of-the-art baselines on synthetic neural populations generated by a Lorenz attractor.
  arXiv  Detail & Related papers  (2025-07-15T17:57:48Z)
• Allostatic Control of Persistent States in Spiking Neural Networks for perception and computation [79.16635054977068]
  We introduce a novel model for updating perceptual beliefs about the environment by extending the concept of Allostasis to the control of internal representations.<n>In this paper, we focus on an application in numerical cognition, where a bump of activity in an attractor network is used as a spatial numerical representation.
  arXiv  Detail & Related papers  (2025-03-20T12:28:08Z)
• A Spacetime Perspective on Dynamical Computation in Neural Information Processing Systems [43.02233537621737]
  We introduce a new'spacetime' perspective on neural computation. We show that temporal dynamics may be by which natural neural systems encode approximate visual quantities.
  arXiv  Detail & Related papers  (2024-09-20T17:25:37Z)
• Learning Delays Through Gradients and Structure: Emergence of Spatiotemporal Patterns in Spiking Neural Networks [0.06752396542927405]
  We present a Spiking Neural Network (SNN) model that incorporates learnable synaptic delays through two approaches. In the latter approach, the network selects and prunes connections, optimizing the delays in sparse connectivity settings. Our results demonstrate the potential of combining delay learning with dynamic pruning to develop efficient SNN models for temporal data processing.
  arXiv  Detail & Related papers  (2024-07-07T11:55:48Z)
• Gradient Starvation: A Learning Proclivity in Neural Networks [97.02382916372594]
  Gradient Starvation arises when cross-entropy loss is minimized by capturing only a subset of features relevant for the task. This work provides a theoretical explanation for the emergence of such feature imbalance in neural networks.
  arXiv  Detail & Related papers  (2020-11-18T18:52:08Z)
• Hyperbolic Neural Networks++ [66.16106727715061]
  We generalize the fundamental components of neural networks in a single hyperbolic geometry model, namely, the Poincar'e ball model. Experiments show the superior parameter efficiency of our methods compared to conventional hyperbolic components, and stability and outperformance over their Euclidean counterparts.
  arXiv  Detail & Related papers  (2020-06-15T08:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.