Continuous Time Continuous Space Homeostatic Reinforcement Learning (CTCS-HRRL) : Towards Biological Self-Autonomous Agent

• URL: http://arxiv.org/abs/2401.08999v1
• Date: Wed, 17 Jan 2024 06:29:34 GMT
• Title: Continuous Time Continuous Space Homeostatic Reinforcement Learning (CTCS-HRRL) : Towards Biological Self-Autonomous Agent
• Authors: Hugo Laurencon, Yesoda Bhargava, Riddhi Zantye, Charbel-Rapha\"el S\'egerie, Johann Lussange, Veeky Baths, Boris Gutkin
• Abstract summary: Homeostasis is a process by which living beings maintain their internal balance. Homeostatic Regulated Reinforcement Learning (HRRL) framework attempts to explain this learned homeostatic behaviour. In this work, we advance the HRRL framework to a continuous time-space environment and validate the CTCS-HRRL framework.
• Score: 0.12068041242343093
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Homeostasis is a biological process by which living beings maintain their internal balance. Previous research suggests that homeostasis is a learned behaviour. Recently introduced Homeostatic Regulated Reinforcement Learning (HRRL) framework attempts to explain this learned homeostatic behavior by linking Drive Reduction Theory and Reinforcement Learning. This linkage has been proven in the discrete time-space, but not in the continuous time-space. In this work, we advance the HRRL framework to a continuous time-space environment and validate the CTCS-HRRL (Continuous Time Continuous Space HRRL) framework. We achieve this by designing a model that mimics the homeostatic mechanisms in a real-world biological agent. This model uses the Hamilton-Jacobian Bellman Equation, and function approximation based on neural networks and Reinforcement Learning. Through a simulation-based experiment we demonstrate the efficacy of this model and uncover the evidence linked to the agent's ability to dynamically choose policies that favor homeostasis in a continuously changing internal-state milieu. Results of our experiments demonstrate that agent learns homeostatic behaviour in a CTCS environment, making CTCS-HRRL a promising framework for modellng animal dynamics and decision-making.

Related papers

• A Dynamic Systems Approach to Modelling Human-Machine Rhythm Interaction [4.33608942673382]
  This study introduces a computational model inspired by the physical and biological processes underlying rhythm processing. Our findings demonstrate the model's ability to accurately perceive and adapt to rhythmic patterns within the human perceptible range.
  arXiv  Detail & Related papers  (2024-06-26T10:07:20Z)
• Interpretable Spatio-Temporal Embedding for Brain Structural-Effective Network with Ordinary Differential Equation [56.34634121544929]
  In this study, we first construct the brain-effective network via the dynamic causal model. We then introduce an interpretable graph learning framework termed Spatio-Temporal Embedding ODE (STE-ODE) This framework incorporates specifically designed directed node embedding layers, aiming at capturing the dynamic interplay between structural and effective networks.
  arXiv  Detail & Related papers  (2024-05-21T20:37:07Z)
• Emergence of Adaptive Circadian Rhythms in Deep Reinforcement Learning [0.0]
  Adapting to regularities of the environment is critical for biological organisms to anticipate events and plan. We study the emergence of circadian-like rhythms in deep reinforcement learning agents.
  arXiv  Detail & Related papers  (2023-07-22T18:47:18Z)
• Real-Time Variational Method for Learning Neural Trajectory and its Dynamics [7.936841911281107]
  We introduce the exponential family variational Kalman filter (eVKF), an online Bayesian method aimed at inferring latent trajectories while simultaneously learning the dynamical system generating them. We derive a closed-form variational analogue to the predict step of the Kalman filter which leads to a provably tighter bound on the ELBO compared to another online variational method. We validate our method on synthetic and real-world data, and, notably, show that it achieves competitive performance.
  arXiv  Detail & Related papers  (2023-05-18T19:52:46Z)
• DriPP: Driven Point Processes to Model Stimuli Induced Patterns in M/EEG Signals [62.997667081978825]
  We develop a novel statistical point process model-called driven temporal point processes (DriPP) We derive a fast and principled expectation-maximization (EM) algorithm to estimate the parameters of this model. Results on standard MEG datasets demonstrate that our methodology reveals event-related neural responses.
  arXiv  Detail & Related papers  (2021-12-08T13:07:21Z)
• Continuous Homeostatic Reinforcement Learning for Self-Regulated Autonomous Agents [0.0]
  We propose an extension of the homeostatic reinforcement learning theory to a continuous environment in space and time. Inspired by the self-regulating mechanisms abundantly present in biology, we also introduce a model for the dynamics of the agent internal state.
  arXiv  Detail & Related papers  (2021-09-14T11:03:58Z)
• Deep Bayesian Active Learning for Accelerating Stochastic Simulation [74.58219903138301]
  Interactive Neural Process (INP) is a deep active learning framework for simulations and with active learning approaches. For active learning, we propose a novel acquisition function, Latent Information Gain (LIG), calculated in the latent space of NP based models. The results demonstrate STNP outperforms the baselines in the learning setting and LIG achieves the state-of-the-art for active learning.
  arXiv  Detail & Related papers  (2021-06-05T01:31:51Z)
• Continuous Learning and Adaptation with Membrane Potential and Activation Threshold Homeostasis [91.3755431537592]
  This paper presents the Membrane Potential and Activation Threshold Homeostasis (MPATH) neuron model. The model allows neurons to maintain a form of dynamic equilibrium by automatically regulating their activity when presented with input. Experiments demonstrate the model's ability to adapt to and continually learn from its input.
  arXiv  Detail & Related papers  (2021-04-22T04:01:32Z)
• GEM: Group Enhanced Model for Learning Dynamical Control Systems [78.56159072162103]
  We build effective dynamical models that are amenable to sample-based learning. We show that learning the dynamics on a Lie algebra vector space is more effective than learning a direct state transition model. This work sheds light on a connection between learning of dynamics and Lie group properties, which opens doors for new research directions.
  arXiv  Detail & Related papers  (2021-04-07T01:08:18Z)
• Training Generative Adversarial Networks by Solving Ordinary Differential Equations [54.23691425062034]
  We study the continuous-time dynamics induced by GAN training. From this perspective, we hypothesise that instabilities in training GANs arise from the integration error. We experimentally verify that well-known ODE solvers (such as Runge-Kutta) can stabilise training.
  arXiv  Detail & Related papers  (2020-10-28T15:23:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.