Emergent time-keeping mechanisms in a deep reinforcement learning agent performing an interval timing task

• URL: http://arxiv.org/abs/2508.15784v2
• Date: Tue, 26 Aug 2025 17:56:42 GMT
• Title: Emergent time-keeping mechanisms in a deep reinforcement learning agent performing an interval timing task
• Authors: Amrapali Pednekar, Alvaro Garrido, Pieter Simoens, Yara Khaluf,
• Abstract summary: We investigate temporal processing in a Deep Reinforcement Learning (DRL) agent performing an interval timing task.<n>Analysis of the agent's internal states revealed neural activations, a ubiquitous pattern in biological systems.<n> Parallels are drawn between the agent's time-keeping strategy and the Striatal Beat Frequency (SBF) model.
• Score: 4.398170461093707
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Drawing parallels between Deep Artificial Neural Networks (DNNs) and biological systems can aid in understanding complex biological mechanisms that are difficult to disentangle. Temporal processing, an extensively researched topic, is one such example that lacks a coherent understanding of its underlying mechanisms. In this study, we investigate temporal processing in a Deep Reinforcement Learning (DRL) agent performing an interval timing task and explore potential biological counterparts to its emergent behavior. The agent was successfully trained to perform a duration production task, which involved marking successive occurrences of a target interval while viewing a video sequence. Analysis of the agent's internal states revealed oscillatory neural activations, a ubiquitous pattern in biological systems. Interestingly, the agent's actions were predominantly influenced by neurons exhibiting these oscillations with high amplitudes and frequencies corresponding to the target interval. Parallels are drawn between the agent's time-keeping strategy and the Striatal Beat Frequency (SBF) model, a biologically plausible model of interval timing. Furthermore, the agent maintained its oscillatory representations and task performance when tested on different video sequences (including a blank video). Thus, once learned, the agent internalized its time-keeping mechanism and showed minimal reliance on its environment to perform the timing task. A hypothesis about the resemblance between this emergent behavior and certain aspects of the evolution of biological processes like circadian rhythms, has been discussed. This study aims to contribute to recent research efforts of utilizing DNNs to understand biological systems, with a particular emphasis on temporal processing.

Related papers

• Progressive Multi-Agent Reasoning for Biological Perturbation Prediction [32.71169480836875]
  We present LINCSQA, a novel benchmark for predicting target gene regulation under complex chemical perturbations.<n>We also propose PBio-Agent, a multi-agent framework that integrates difficulty-aware task sequencing with iterative knowledge refinement.<n>Our key insight is that genes affected by the same perturbation share causal structure, allowing confidently predicted genes to contextualize more challenging cases.
  arXiv  Detail & Related papers  (2026-02-07T06:59:44Z)
• Causal Representation Meets Stochastic Modeling under Generic Geometry [49.24293444627916]
  We develop causal representation learning for continuous-time latent point processes.<n>We develop MUTATE, an identifiable variational autoencoder framework with a time-adaptive transition module.<n>Across simulated and empirical studies, we find that MUTATE can effectively answer scientific questions.
  arXiv  Detail & Related papers  (2026-02-04T20:40:53Z)
• Unleashing Temporal Capacity of Spiking Neural Networks through Spatiotemporal Separation [67.69345363409835]
  Spiking Neural Networks (SNNs) are considered naturally suited for temporal processing, with membrane potential propagation widely regarded as the core temporal modeling mechanism.<n>We design Non-Stateful (NS) models progressively removing membrane propagation to its stage-wise role. Experiments reveal a counterintuitive phenomenon: moderate removal in shallow layers improves performance, while excessive removal causes collapse.
  arXiv  Detail & Related papers  (2025-12-05T07:05:53Z)
• Modulation of temporal decision-making in a deep reinforcement learning agent under the dual-task paradigm [4.398170461093707]
  This study explores the interference in temporal processing within a dual-task paradigm from an artificial intelligence (AI) perspective.<n>Two deep reinforcement learning (DRL) agents were separately trained for each of these tasks.<n>These agents exhibited emergent behavior consistent with human timing research.
  arXiv  Detail & Related papers  (2025-11-03T10:09:55Z)
• Synchronization and semantization in deep spiking networks [0.9411751957919126]
  Recent studies have shown how spiking networks can learn complex functionality through error-correcting plasticity, but the resulting structures and dynamics remain poorly studied.<n>We train a multi-layer spiking network, as a conceptual analog of visual hierarchy, for visual input classification using spike-time encoding.<n>After learning, we observe the development of distinct-temporal activity patterns. While input patterns are synchronous by construction, activity in early layers first spreads out over time, followed by re-convergence into sharp pulses as classes are gradually extracted.<n>The emergence of synchronicity is accompanied by the formation of increasingly distinct pathways, reflecting the gradual semantization
  arXiv  Detail & Related papers  (2025-08-18T14:51:58Z)
• State Space Models Naturally Produce Traveling Waves, Time Cells, and Scale to Abstract Cognitive Functions [7.097247619177705]
  We propose a framework based on State-Space Models (SSMs), an emerging class of deep learning architectures.<n>We demonstrate that the model spontaneously develops neural representations that strikingly mimic biological 'time cells'<n>Our findings position SSMs as a compelling framework that connects single-neuron dynamics to cognitive phenomena.
  arXiv  Detail & Related papers  (2025-07-18T03:53:16Z)
• Inferring the time-varying coupling of dynamical systems with temporal convolutional autoencoders [0.0]
  We introduce Temporal Autoencoders for Causal Inference (TACI) TACI combines a new surrogate data metric for assessing causal interactions with a novel two-headed machine learning architecture. We demonstrate TACI's ability to accurately quantify dynamic causal interactions across a variety of systems.
  arXiv  Detail & Related papers  (2024-06-05T12:51:20Z)
• LINOCS: Lookahead Inference of Networked Operators for Continuous Stability [4.508868068781057]
  We introduce Lookahead-driven Inference of Networked Operators for Continuous Stability (LINOCS) LINOCS is a robust learning procedure for identifying hidden dynamical interactions in noisy time-series data. We demonstrate LINOCS' ability to recover the ground truth dynamical operators underlying synthetic time-series data.
  arXiv  Detail & Related papers  (2024-04-28T18:16:58Z)
• TC-LIF: A Two-Compartment Spiking Neuron Model for Long-Term Sequential Modelling [54.97005925277638]
  The identification of sensory cues associated with potential opportunities and dangers is frequently complicated by unrelated events that separate useful cues by long delays. It remains a challenging task for state-of-the-art spiking neural networks (SNNs) to establish long-term temporal dependency between distant cues. We propose a novel biologically inspired Two-Compartment Leaky Integrate-and-Fire spiking neuron model, dubbed TC-LIF.
  arXiv  Detail & Related papers  (2023-08-25T08:54:41Z)
• Emergence of Adaptive Circadian Rhythms in Deep Reinforcement Learning [0.0]
  Adapting to regularities of the environment is critical for biological organisms to anticipate events and plan. We study the emergence of circadian-like rhythms in deep reinforcement learning agents.
  arXiv  Detail & Related papers  (2023-07-22T18:47:18Z)
• Long Short-term Memory with Two-Compartment Spiking Neuron [64.02161577259426]
  We propose a novel biologically inspired Long Short-Term Memory Leaky Integrate-and-Fire spiking neuron model, dubbed LSTM-LIF. Our experimental results, on a diverse range of temporal classification tasks, demonstrate superior temporal classification capability, rapid training convergence, strong network generalizability, and high energy efficiency of the proposed LSTM-LIF model. This work, therefore, opens up a myriad of opportunities for resolving challenging temporal processing tasks on emerging neuromorphic computing machines.
  arXiv  Detail & Related papers  (2023-07-14T08:51:03Z)
• Reducing Catastrophic Forgetting in Self Organizing Maps with Internally-Induced Generative Replay [67.50637511633212]
  A lifelong learning agent is able to continually learn from potentially infinite streams of pattern sensory data. One major historic difficulty in building agents that adapt is that neural systems struggle to retain previously-acquired knowledge when learning from new samples. This problem is known as catastrophic forgetting (interference) and remains an unsolved problem in the domain of machine learning to this day.
  arXiv  Detail & Related papers  (2021-12-09T07:11:14Z)
• Consistency of mechanistic causal discovery in continuous-time using Neural ODEs [85.7910042199734]
  We consider causal discovery in continuous-time for the study of dynamical systems. We propose a causal discovery algorithm based on penalized Neural ODEs.
  arXiv  Detail & Related papers  (2021-05-06T08:48:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.