Connectivity structure and dynamics of nonlinear recurrent neural networks

• URL: http://arxiv.org/abs/2409.01969v3
• Date: Mon, 27 Oct 2025 01:18:24 GMT
• Title: Connectivity structure and dynamics of nonlinear recurrent neural networks
• Authors: David G. Clark, Owen Marschall, Alexander van Meegen, Ashok Litwin-Kumar,
• Abstract summary: We develop a theory to analyze how connectivity structure shape high-dimensional collective activity in neural networks.<n>We show that connectivity structure can be invisible in single-neuron activities while dramatically shaping collective activity.
• Score: 41.038855001225826
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Studies of the dynamics of nonlinear recurrent neural networks often assume independent and identically distributed couplings, but large-scale connectomics data indicate that biological neural circuits exhibit markedly different connectivity properties. These include rapidly decaying singular-value spectra and structured singular-vector overlaps. Here, we develop a theory to analyze how these forms of structure shape high-dimensional collective activity in nonlinear recurrent neural networks. We first introduce the random-mode model, a random-matrix ensemble related to the singular-value decomposition that enables control over the spectrum and right-left mode overlaps. Then, using a novel path-integral calculation, we derive analytical expressions that reveal how connectivity structure affects features of collective dynamics: the dimension of activity, which quantifies the number of high-variance collective-activity fluctuations, and the temporal correlations that characterize the timescales of these fluctuations. We show that connectivity structure can be invisible in single-neuron activities while dramatically shaping collective activity. Furthermore, despite the nonlinear, high-dimensional nature of these networks, the dimension of activity depends on just two connectivity parameters -- the variance of the couplings and the effective rank of the coupling matrix, which quantifies the number of dominant rank-one connectivity components. We contrast the effects of single-neuron heterogeneity and low dimensional connectivity, making predictions about how z-scoring data affects the dimension of activity. Finally, we demonstrate the presence of structured overlaps between left and right modes in the Drosophila connectome, incorporate them into the theory, and show how they further shape collective dynamics.

Related papers

• Drift-Diffusion Matching: Embedding dynamics in latent manifolds of asymmetric neural networks [0.8793721044482612]
  We introduce a general framework for training continuous-time RNNs to represent arbitrary dynamical systems within a low-dimensional latent subspace.<n>We show that RNNs can faithfully embed the drift and diffusion of a given differential equation, including nonlinear and nonequilibrium dynamics such as chaotic attractors.<n>Our results extend attractor neural network theory beyond equilibrium, showing that asymmetric neural populations can implement a broad class of dynamical computations within low-dimensional, unifying ideas from associative memory, nonequilibrium statistical mechanics, and neural computation.
  arXiv  Detail & Related papers  (2026-02-16T16:15:59Z)
• Krylov complexity in ergodically constrained nonintegrable transverse-field Ising model [0.0]
  We partition the chain into two equal segments within which the spins interact with different coupling strengths.<n>The ratio of these couplings defines an inhomogeneity parameter, whose variation away from unity leads to constrained dynamics.<n>We further examine the consequences for operator growth in Krylov space and for entanglement generation in the system's eigenstates.
  arXiv  Detail & Related papers  (2025-12-23T11:50:01Z)
• Topology Identification and Inference over Graphs [61.06365536861156]
  Topology identification and inference of processes evolving over graphs arise in timely applications involving brain, transportation, financial, power, as well as social and information networks.<n>This chapter provides an overview of graph topology identification and statistical inference methods for multidimensional data.
  arXiv  Detail & Related papers  (2025-12-11T00:47:09Z)
• Statistics of correlations in nonlinear recurrent neural networks [0.0]
  We derive exact expressions for the statistics of correlations of nonlinear recurrent networks in the limit of a large number of neurons.<n>We present explicit results for power-law activations, revealing scaling behavior controlled by the network coupling.<n>In addition, we introduce a class of activation functions based on Pade approximants and provide analytic predictions for their correlation statistics.
  arXiv  Detail & Related papers  (2025-10-06T19:12:58Z)
• Biologically Inspired Spiking Diffusion Model with Adaptive Lateral Selection Mechanism [5.135901078097114]
  We develop a novel diffusion model based on spiking neural networks (SNNs) We leverage this spiking inner loop alongside a lateral connection mechanism to iteratively refine the substructure selection network. Our model consistently surpasses state-of-the-art SNN-based generative models across multiple benchmark datasets.
  arXiv  Detail & Related papers  (2025-03-31T06:31:50Z)
• Exploring the Nexus of Many-Body Theories through Neural Network Techniques: the Tangent Model [4.955567578189708]
  We present a physically informed neural network representation of the effective interactions associated with coupled-cluster downfolding models. The representation allows us to evaluate the effective interactions efficiently for various geometrical configurations of chemical systems.
  arXiv  Detail & Related papers  (2025-01-27T05:36:55Z)
• Identifying the impact of local connectivity patterns on dynamics in excitatory-inhibitory networks [4.913318028439159]
  We show that a particular pattern of connectivity, chain motifs, have a much stronger impact on dominant eigenmodes than other pairwise motifs.<n>An overrepresentation of chain motifs induces a strong positive eigenvalue in inhibition-dominated networks.<n>These findings have direct implications for the interpretation of experiments in which responses to optogenetic perturbations are measured and used to infer the dynamical regime of cortical circuits.
  arXiv  Detail & Related papers  (2024-11-11T08:57:44Z)
• Integrating GNN and Neural ODEs for Estimating Non-Reciprocal Two-Body Interactions in Mixed-Species Collective Motion [0.0]
  We present a novel deep learning framework for estimating the underlying equations of motion from observed trajectories. Our framework integrates graph neural networks with neural differential equations, enabling effective prediction of two-body interactions.
  arXiv  Detail & Related papers  (2024-05-26T09:47:17Z)
• Interpretable Spatio-Temporal Embedding for Brain Structural-Effective Network with Ordinary Differential Equation [56.34634121544929]
  In this study, we first construct the brain-effective network via the dynamic causal model. We then introduce an interpretable graph learning framework termed Spatio-Temporal Embedding ODE (STE-ODE) This framework incorporates specifically designed directed node embedding layers, aiming at capturing the dynamic interplay between structural and effective networks.
  arXiv  Detail & Related papers  (2024-05-21T20:37:07Z)
• DSAM: A Deep Learning Framework for Analyzing Temporal and Spatial Dynamics in Brain Networks [4.041732967881764]
  Most rs-fMRI studies compute a single static functional connectivity matrix across brain regions of interest. These approaches are at risk of oversimplifying brain dynamics and lack proper consideration of the goal at hand. We propose a novel interpretable deep learning framework that learns goal-specific functional connectivity matrix directly from time series.
  arXiv  Detail & Related papers  (2024-05-19T23:35:06Z)
• Demolition and Reinforcement of Memories in Spin-Glass-like Neural Networks [0.0]
  The aim of this thesis is to understand the effectiveness of Unlearning in both associative memory models and generative models. The selection of structured data enables an associative memory model to retrieve concepts as attractors of a neural dynamics with considerable basins of attraction. A novel regularization technique for Boltzmann Machines is presented, proving to outperform previously developed methods in learning hidden probability distributions from data-sets.
  arXiv  Detail & Related papers  (2024-03-04T23:12:42Z)
• On the Trade-off Between Efficiency and Precision of Neural Abstraction [62.046646433536104]
  Neural abstractions have been recently introduced as formal approximations of complex, nonlinear dynamical models. We employ formal inductive synthesis procedures to generate neural abstractions that result in dynamical models with these semantics.
  arXiv  Detail & Related papers  (2023-07-28T13:22:32Z)
• Equivalence of Additive and Multiplicative Coupling in Spiking Neural Networks [0.0]
  Spiking neural network models characterize the emergent collective dynamics of circuits of biological neurons. We show that spiking neural network models with additive coupling are equivalent to models with multiplicative coupling.
  arXiv  Detail & Related papers  (2023-03-31T20:19:11Z)
• Order-invariant two-photon quantum correlations in PT-symmetric interferometers [62.997667081978825]
  Multiphoton correlations in linear photonic quantum networks are governed by matrix permanents. We show that the overall multiphoton behavior of a network from its individual building blocks typically defies intuition. Our results underline new ways in which quantum correlations may be preserved in counterintuitive ways even in small-scale non-Hermitian networks.
  arXiv  Detail & Related papers  (2023-02-23T09:43:49Z)
• Dimension of activity in random neural networks [6.752538702870792]
  Neural networks are high-dimensional nonlinear dynamical systems that process information through the coordinated activity of many connected units. We calculate cross covariances self-consistently via a two-site cavity DMFT. Our formulae apply to a wide range of single-unit dynamics and generalize to non-i.i.d. couplings.
  arXiv  Detail & Related papers  (2022-07-25T17:38:21Z)
• Learning the Evolutionary and Multi-scale Graph Structure for Multivariate Time Series Forecasting [50.901984244738806]
  We show how to model the evolutionary and multi-scale interactions of time series. In particular, we first provide a hierarchical graph structure cooperated with the dilated convolution to capture the scale-specific correlations. A unified neural network is provided to integrate the components above to get the final prediction.
  arXiv  Detail & Related papers  (2022-06-28T08:11:12Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Learning Neural Causal Models with Active Interventions [83.44636110899742]
  We introduce an active intervention-targeting mechanism which enables a quick identification of the underlying causal structure of the data-generating process. Our method significantly reduces the required number of interactions compared with random intervention targeting. We demonstrate superior performance on multiple benchmarks from simulated to real-world data.
  arXiv  Detail & Related papers  (2021-09-06T13:10:37Z)
• Convolutional Filtering and Neural Networks with Non Commutative Algebras [153.20329791008095]
  We study the generalization of non commutative convolutional neural networks. We show that non commutative convolutional architectures can be stable to deformations on the space of operators.
  arXiv  Detail & Related papers  (2021-08-23T04:22:58Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.