Ornstein-Uhlenbeck Adaptation as a Mechanism for Learning in Brains and Machines

• URL: http://arxiv.org/abs/2410.13563v2
• Date: Wed, 23 Oct 2024 13:19:26 GMT
• Title: Ornstein-Uhlenbeck Adaptation as a Mechanism for Learning in Brains and Machines
• Authors: Jesus Garcia Fernandez, Nasir Ahmad, Marcel van Gerven,
• Abstract summary: We introduce a novel approach that leverages noise in the parameters of the system and global reinforcement signals. operating in continuous time, Orstein-Uhlenbeck adaptation (OUA) is proposed as a general mechanism for learning. OUA provides a viable alternative to traditional gradient-based methods, with potential applications in neuromorphic computing.
• Score: 2.1168858935852013
• License:
• Abstract: Learning is a fundamental property of intelligent systems, observed across biological organisms and engineered systems. While modern intelligent systems typically rely on gradient descent for learning, the need for exact gradients and complex information flow makes its implementation in biological and neuromorphic systems challenging. This has motivated the exploration of alternative learning mechanisms that can operate locally and do not rely on exact gradients. In this work, we introduce a novel approach that leverages noise in the parameters of the system and global reinforcement signals. Using an Ornstein-Uhlenbeck process with adaptive dynamics, our method balances exploration and exploitation during learning, driven by deviations from error predictions, akin to reward prediction error. Operating in continuous time, Orstein-Uhlenbeck adaptation (OUA) is proposed as a general mechanism for learning dynamic, time-evolving environments. We validate our approach across diverse tasks, including supervised learning and reinforcement learning in feedforward and recurrent systems. Additionally, we demonstrate that it can perform meta-learning, adjusting hyper-parameters autonomously. Our results indicate that OUA provides a viable alternative to traditional gradient-based methods, with potential applications in neuromorphic computing. It also hints at a possible mechanism for noise-driven learning in the brain, where stochastic neurotransmitter release may guide synaptic adjustments.

Related papers

• Let's do the time-warp-attend: Learning topological invariants of dynamical systems [3.9735602856280132]
  We propose a data-driven, physically-informed deep-learning framework for classifying dynamical regimes and characterizing bifurcation boundaries. We focus on the paradigmatic case of the supercritical Hopf bifurcation, which is used to model periodic dynamics across a range of applications. Our method provides valuable insights into the qualitative, long-term behavior of a wide range of dynamical systems, and can detect bifurcations or catastrophic transitions in large-scale physical and biological systems.
  arXiv  Detail & Related papers  (2023-12-14T18:57:16Z)
• Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment [65.268245109828]
  We examine algorithms for conducting credit assignment in artificial neural networks that are inspired or motivated by neurobiology. We organize the ever-growing set of brain-inspired learning schemes into six general families and consider these in the context of backpropagation of errors. The results of this review are meant to encourage future developments in neuro-mimetic systems and their constituent learning processes.
  arXiv  Detail & Related papers  (2023-12-01T05:20:57Z)
• Learning low-dimensional dynamics from whole-brain data improves task capture [2.82277518679026]
  We introduce a novel approach to learning low-dimensional approximations of neural dynamics by using a sequential variational autoencoder (SVAE) Our method finds smooth dynamics that can predict cognitive processes with accuracy higher than classical methods. We evaluate our approach on various task-fMRI datasets, including motor, working memory, and relational processing tasks.
  arXiv  Detail & Related papers  (2023-05-18T18:43:13Z)
• Incremental procedural and sensorimotor learning in cognitive humanoid robots [52.77024349608834]
  This work presents a cognitive agent that can learn procedures incrementally. We show the cognitive functions required in each substage and how adding new functions helps address tasks previously unsolved by the agent. Results show that this approach is capable of solving complex tasks incrementally.
  arXiv  Detail & Related papers  (2023-04-30T22:51:31Z)
• The least-control principle for learning at equilibrium [65.2998274413952]
  We present a new principle for learning equilibrium recurrent neural networks, deep equilibrium models, or meta-learning. Our results shed light on how the brain might learn and offer new ways of approaching a broad class of machine learning problems.
  arXiv  Detail & Related papers  (2022-07-04T11:27:08Z)
• Agnostic Physics-Driven Deep Learning [82.89993762912795]
  This work establishes that a physical system can perform statistical gradient learning without gradient computations. In Aeqprop, the specifics of the system do not have to be known: the procedure is based on external manipulations. Aeqprop also establishes that in natural (bio)physical systems, genuine gradient-based statistical learning may result from generic, relatively simple mechanisms.
  arXiv  Detail & Related papers  (2022-05-30T12:02:53Z)
• Neurosymbolic hybrid approach to driver collision warning [64.02492460600905]
  There are two main algorithmic approaches to autonomous driving systems. Deep learning alone has achieved state-of-the-art results in many areas. But sometimes it can be very difficult to debug if the deep learning model doesn't work.
  arXiv  Detail & Related papers  (2022-03-28T20:29:50Z)
• Using Data Assimilation to Train a Hybrid Forecast System that Combines Machine-Learning and Knowledge-Based Components [52.77024349608834]
  We consider the problem of data-assisted forecasting of chaotic dynamical systems when the available data is noisy partial measurements. We show that by using partial measurements of the state of the dynamical system, we can train a machine learning model to improve predictions made by an imperfect knowledge-based model.
  arXiv  Detail & Related papers  (2021-02-15T19:56:48Z)
• Inferring Global Dynamics Using a Learning Machine [5.07635313657742]
  We show that by using a learning machine we can achieve such a goal to a certain extent. It is found that following an appropriate training strategy that monotonously decreases the cost function, the learning machine in different training stage can mimic the system at different parameter set.
  arXiv  Detail & Related papers  (2020-09-28T02:54:44Z)
• Planning from Images with Deep Latent Gaussian Process Dynamics [2.924868086534434]
  Planning is a powerful approach to control problems with known environment dynamics. In unknown environments the agent needs to learn a model of the system dynamics to make planning applicable. We propose to learn a deep latent Gaussian process dynamics (DLGPD) model that learns low-dimensional system dynamics from environment interactions with visual observations.
  arXiv  Detail & Related papers  (2020-05-07T21:29:45Z)
• Uncertainty-based Modulation for Lifelong Learning [1.3334365645271111]
  We present an algorithm inspired by neuromodulatory mechanisms in the human brain that integrates and expands upon Stephen Grossberg's Adaptive Resonance Theory proposals. Specifically, it builds on the concept of uncertainty, and employs a series of neuromodulatory mechanisms to enable continuous learning. We demonstrate the critical role of developing these systems in a closed-loop manner where the environment and the agent's behaviors constrain and guide the learning process.
  arXiv  Detail & Related papers  (2020-01-27T14:34:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.