Transient Dynamics in Lattices of Differentiating Ring Oscillators

• URL: http://arxiv.org/abs/2506.07253v1
• Date: Sun, 08 Jun 2025 18:29:15 GMT
• Title: Transient Dynamics in Lattices of Differentiating Ring Oscillators
• Authors: Peter DelMastro, Arjun Karuvally, Hananel Hazan, Hava Siegelmann, Edward Rietman,
• Abstract summary: Recurrent neural networks (RNNs) are machine learning models widely used for learning temporal relationships.<n>We show via numerical simulation that large lattices of differentiating neuron rings exhibit local neural synchronization behavior.
• Score: 0.34952465649465553
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recurrent neural networks (RNNs) are machine learning models widely used for learning temporal relationships. Current state-of-the-art RNNs use integrating or spiking neurons -- two classes of computing units whose outputs depend directly on their internal states -- and accordingly there is a wealth of literature characterizing the behavior of large networks built from these neurons. On the other hand, past research on differentiating neurons, whose outputs are computed from the derivatives of their internal states, remains limited to small hand-designed networks with fewer than one-hundred neurons. Here we show via numerical simulation that large lattices of differentiating neuron rings exhibit local neural synchronization behavior found in the Kuramoto model of interacting oscillators. We begin by characterizing the periodic orbits of uncoupled rings, herein called ring oscillators. We then show the emergence of local correlations between oscillators that grow over time when these rings are coupled together into lattices. As the correlation length grows, transient dynamics arise in which large regions of the lattice settle to the same periodic orbit, and thin domain boundaries separate adjacent, out-of-phase regions. The steady-state scale of these correlated regions depends on how the neurons are shared between adjacent rings, which suggests that lattices of differentiating ring oscillator might be tuned to be used as reservoir computers. Coupled with their simple circuit design and potential for low-power consumption, differentiating neural nets therefore represent a promising substrate for neuromorphic computing that will enable low-power AI applications.

Related papers

• Langevin Flows for Modeling Neural Latent Dynamics [81.81271685018284]
  We introduce LangevinFlow, a sequential Variational Auto-Encoder where the time evolution of latent variables is governed by the underdamped Langevin equation.<n>Our approach incorporates physical priors -- such as inertia, damping, a learned potential function, and forces -- to represent both autonomous and non-autonomous processes in neural systems.<n>Our method outperforms state-of-the-art baselines on synthetic neural populations generated by a Lorenz attractor.
  arXiv  Detail & Related papers  (2025-07-15T17:57:48Z)
• Allostatic Control of Persistent States in Spiking Neural Networks for perception and computation [79.16635054977068]
  We introduce a novel model for updating perceptual beliefs about the environment by extending the concept of Allostasis to the control of internal representations.<n>In this paper, we focus on an application in numerical cognition, where a bump of activity in an attractor network is used as a spatial numerical representation.
  arXiv  Detail & Related papers  (2025-03-20T12:28:08Z)
• Confidence Regulation Neurons in Language Models [91.90337752432075]
  This study investigates the mechanisms by which large language models represent and regulate uncertainty in next-token predictions. Entropy neurons are characterized by an unusually high weight norm and influence the final layer normalization (LayerNorm) scale to effectively scale down the logits. token frequency neurons, which we describe here for the first time, boost or suppress each token's logit proportionally to its log frequency, thereby shifting the output distribution towards or away from the unigram distribution.
  arXiv  Detail & Related papers  (2024-06-24T01:31:03Z)
• Astrocytes as a mechanism for meta-plasticity and contextually-guided network function [2.66269503676104]
  Astrocytes are a ubiquitous and enigmatic type of non-neuronal cell. Astrocytes may play a more direct and active role in brain function and neural computation.
  arXiv  Detail & Related papers  (2023-11-06T20:31:01Z)
• The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks [64.08042492426992]
  We introduce the Expressive Memory (ELM) neuron model, a biologically inspired model of a cortical neuron. Our ELM neuron can accurately match the aforementioned input-output relationship with under ten thousand trainable parameters. We evaluate it on various tasks with demanding temporal structures, including the Long Range Arena (LRA) datasets.
  arXiv  Detail & Related papers  (2023-06-14T13:34:13Z)
• Equivalence of Additive and Multiplicative Coupling in Spiking Neural Networks [0.0]
  Spiking neural network models characterize the emergent collective dynamics of circuits of biological neurons. We show that spiking neural network models with additive coupling are equivalent to models with multiplicative coupling.
  arXiv  Detail & Related papers  (2023-03-31T20:19:11Z)
• Phenomenological Model of Superconducting Optoelectronic Loop Neurons [0.0]
  Superconducting optoelectronic loop neurons are a class of circuits potentially conducive to networks for large-scale artificial cognition. To date, all simulations of loop neurons have used first-principles circuit analysis to model the behavior of synapses, dendrites, and neurons. Here we introduce a modeling framework that captures the behavior of the relevant synaptic, dendritic, and neuronal circuits.
  arXiv  Detail & Related papers  (2022-10-18T16:38:35Z)
• Limits of Entrainment of Circadian Neuronal Networks [0.0]
  Circadian rhythmicity lies at the center of various important physiological and behavioral processes in mammals. We study a modern computational neuroscience model to determine the limits of circadian synchronization to external light signals of different frequency and duty cycle.
  arXiv  Detail & Related papers  (2022-08-23T17:57:21Z)
• Cross-Frequency Coupling Increases Memory Capacity in Oscillatory Neural Networks [69.42260428921436]
  Cross-frequency coupling (CFC) is associated with information integration across populations of neurons. We construct a model of CFC which predicts a computational role for observed $theta - gamma$ oscillatory circuits in the hippocampus and cortex. We show that the presence of CFC increases the memory capacity of a population of neurons connected by plastic synapses.
  arXiv  Detail & Related papers  (2022-04-05T17:13:36Z)
• Spatiotemporal Spike-Pattern Selectivity in Single Mixed-Signal Neurons with Balanced Synapses [0.27998963147546135]
  Mixed-signal neuromorphic processors could be used for inference and learning. We show how inhomogeneous synaptic circuits could be utilized for resource-efficient implementation of network layers.
  arXiv  Detail & Related papers  (2021-06-10T12:04:03Z)
• Continuous Learning and Adaptation with Membrane Potential and Activation Threshold Homeostasis [91.3755431537592]
  This paper presents the Membrane Potential and Activation Threshold Homeostasis (MPATH) neuron model. The model allows neurons to maintain a form of dynamic equilibrium by automatically regulating their activity when presented with input. Experiments demonstrate the model's ability to adapt to and continually learn from its input.
  arXiv  Detail & Related papers  (2021-04-22T04:01:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.