Unifying Dynamical Systems and Graph Theory to Mechanistically Understand Computation in Neural Networks

• URL: http://arxiv.org/abs/2605.03598v2
• Date: Wed, 06 May 2026 09:13:50 GMT
• Title: Unifying Dynamical Systems and Graph Theory to Mechanistically Understand Computation in Neural Networks
• Abstract summary: We show that the spatial and temporal function of recurrent neural networks can be recovered by modelling the network as a graph.<n>We introduce resolvent-RNNs (R-RNNs), which constrain multi-hop pathways and thereby induce temporal sparsity beyond that achieved by standard L1 regularisation.
• Score: 1.644043499620662
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding how biological and artificial neural networks implement computation from connectivity is a central problem in neuroscience and machine learning. In neural systems, structural and functional connectivity are known to diverge, motivating approaches that move beyond direct connections alone. Here, we show that the spatial and temporal function of recurrent neural networks (RNNs) trained on hierarchically modular tasks can be recovered by modelling the network as a graph and analysing the multi-hop pathways between input and output units. In particular, decomposing these pathways by hop length reveals how the network temporally routes information. This perspective reframes regularisation: if function is implemented through multi-hop communication, then standard penalties such as L1 regularisation, which act only on individual weights, constrain single-hop structure rather than the multi-hop pathways that support computation. Motivated by this view, we introduce resolvent-RNNs (R-RNNs), which constrain multi-hop pathways and thereby induce temporal sparsity beyond that achieved by standard L1 regularisation. Compared with L1 regularisation, R-RNNs achieve improved performance by inducing temporal sparsity that matches the task structure, even when the task signal is sparse. Moreover, R-RNNs exhibit stronger sparsity-function alignment, reflected in their increased robustness under strong regularisation. Together, our results identify multi-hop communication as a key principle linking structure to function in recurrent networks, and suggest that sparsity should be defined over functional pathways rather than individual parameters.

Related papers

• Topological Neural Dynamics: A Neuron-wise Framework for Sequence Modeling [55.07411490538404]
  Topological Neural Dynamics (TND) is a sequence modeling framework that shifts from layer-wise to neuron-wise dynamics.<n>TND represents a neural system as a directed neuron graph, an interaction operator, and a local dynamics function.<n>We evaluate TND as a case study on a behavior cloning task in single-player Pong.
  arXiv  Detail & Related papers  (2026-06-19T10:18:24Z)
• Rare Events, Real Signals: Functional Ensembles as Units of Computation in Deep Spiking Networks [0.0]
  We investigate how internal representations emerge across hierarchical processing systems by introducing a neuroscience-inspired framework for analyzing deep spiking neural networks (SNN)<n>We form the first-order functionally-connected (1FC) group of a neuron based on its statistically significant pairwise correlations with neurons from the previous layer of a trained SNN architecture.<n>Our analysis shows that several principles of functional connectivity previously observed in biological cortex are preserved in spiking ResNet architectures.
  arXiv  Detail & Related papers  (2026-05-21T15:44:32Z)
• Cannistraci-Hebb Training on Ultra-Sparse Spiking Neural Networks [10.30800655748035]
  Spiking neural networks (SNNs) inherently possess temporal activation sparsity.<n>Existing methods fail to achieve ultra-sparse network structures without significant performance loss.<n>We propose the Cannistraci-Hebb Spiking Neural Network (CH-SNN), a novel and generalizable dynamic sparse training framework for SNNs.
  arXiv  Detail & Related papers  (2025-11-05T07:59:19Z)
• Hardware-Friendly Implementation of Physical Reservoir Computing with CMOS-based Time-domain Analog Spiking Neurons [0.26963330643873434]
  This paper introduces a spiking neural network (SNN) for a hardware-friendly physical reservoir computing (RC) on a complementary metal-oxide-semiconductor (CMOS) platform. We demonstrate RC through short-term memory and exclusive OR tasks, and the spoken digit recognition task with an accuracy of 97.7%.
  arXiv  Detail & Related papers  (2024-09-18T00:23:00Z)
• A Spiking Binary Neuron -- Detector of Causal Links [0.0]
  Causal relationship recognition is a fundamental operation in neural networks aimed at learning behavior, action planning, and inferring external world dynamics. This research paper presents a novel approach to realize causal relationship recognition using a simple spiking binary neuron.
  arXiv  Detail & Related papers  (2023-09-15T15:34:17Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis [94.64007376939735]
  We theoretically characterize the impact of connectivity patterns on the convergence of deep neural networks (DNNs) under gradient descent training. We show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate.
  arXiv  Detail & Related papers  (2022-05-11T17:43:54Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)
• Theory of gating in recurrent neural networks [5.672132510411465]
  Recurrent neural networks (RNNs) are powerful dynamical models, widely used in machine learning (ML) and neuroscience. Here, we show that gating offers flexible control of two salient features of the collective dynamics. The gate controlling timescales leads to a novel, marginally stable state, where the network functions as a flexible integrator.
  arXiv  Detail & Related papers  (2020-07-29T13:20:58Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• Recurrent Neural Network Learning of Performance and Intrinsic Population Dynamics from Sparse Neural Data [77.92736596690297]
  We introduce a novel training strategy that allows learning not only the input-output behavior of an RNN but also its internal network dynamics. We test the proposed method by training an RNN to simultaneously reproduce internal dynamics and output signals of a physiologically-inspired neural model. Remarkably, we show that the reproduction of the internal dynamics is successful even when the training algorithm relies on the activities of a small subset of neurons.
  arXiv  Detail & Related papers  (2020-05-05T14:16:54Z)
• STDPG: A Spatio-Temporal Deterministic Policy Gradient Agent for Dynamic Routing in SDN [6.27420060051673]
  Dynamic routing in software-defined networking (SDN) can be viewed as a centralized decision-making problem. We propose a novel model-free framework for dynamic routing in SDN, which is referred to as SDN-temporal deterministic policy gradient (STDPG) agent. STDPG achieves better routing solutions in terms of average end-to-end delay.
  arXiv  Detail & Related papers  (2020-04-21T07:19:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.