Learning offline: memory replay in biological and artificial reinforcement learning

• URL: http://arxiv.org/abs/2109.10034v1
• Date: Tue, 21 Sep 2021 08:57:19 GMT
• Title: Learning offline: memory replay in biological and artificial reinforcement learning
• Authors: Emma L. Roscow, Raymond Chua, Rui Ponte Costa, Matt W. Jones, and Nathan Lepora
• Abstract summary: We review the functional roles of replay in the fields of neuroscience and AI. Replay is important for memory consolidation in biological neural networks. It is also key to stabilising learning in deep neural networks.
• Score: 1.0136215038345011
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Learning to act in an environment to maximise rewards is among the brain's key functions. This process has often been conceptualised within the framework of reinforcement learning, which has also gained prominence in machine learning and artificial intelligence (AI) as a way to optimise decision-making. A common aspect of both biological and machine reinforcement learning is the reactivation of previously experienced episodes, referred to as replay. Replay is important for memory consolidation in biological neural networks, and is key to stabilising learning in deep neural networks. Here, we review recent developments concerning the functional roles of replay in the fields of neuroscience and AI. Complementary progress suggests how replay might support learning processes, including generalisation and continual learning, affording opportunities to transfer knowledge across the two fields to advance the understanding of biological and artificial learning and memory.

Related papers

• Sequential Learning in the Dense Associative Memory [1.2289361708127877]
  We investigate the performance of the Dense Associative Memory in sequential learning problems. We show that existing sequential learning methods can be applied to the Dense Associative Memory to improve sequential learning performance.
  arXiv  Detail & Related papers  (2024-09-24T04:23:00Z)
• Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks [74.3099028063756]
  We develop a new method with neuronal operations based on lateral connections and Hebbian learning. We show that Hebbian and anti-Hebbian learning on recurrent lateral connections can effectively extract the principal subspace of neural activities. Our method consistently solves for spiking neural networks with nearly zero forgetting.
  arXiv  Detail & Related papers  (2024-02-19T09:29:37Z)
• Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment [65.268245109828]
  We examine algorithms for conducting credit assignment in artificial neural networks that are inspired or motivated by neurobiology. We organize the ever-growing set of brain-inspired learning schemes into six general families and consider these in the context of backpropagation of errors. The results of this review are meant to encourage future developments in neuro-mimetic systems and their constituent learning processes.
  arXiv  Detail & Related papers  (2023-12-01T05:20:57Z)
• Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits [61.94533459151743]
  This work addresses the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks.
  arXiv  Detail & Related papers  (2023-03-30T02:40:28Z)
• Consciousness is learning: predictive processing systems that learn by binding may perceive themselves as conscious [0.0]
  We argue that a predictive processing system may flexibly generalize in novel situations by forming working memories for perceptions and actions from single examples. We describe how the brain could have evolved to use perceptual value prediction for reinforcement learning of complex action policies simultaneously implementing multiple survival and reproduction strategies. 'Conscious experience' is how such a learning system perceptually represents its own functioning, suggesting an answer to the meta problem of consciousness.
  arXiv  Detail & Related papers  (2023-01-17T17:06:48Z)
• Continual learning benefits from multiple sleep mechanisms: NREM, REM, and Synaptic Downscaling [51.316408685035526]
  Learning new tasks and skills in succession without losing prior learning is a computational challenge for both artificial and biological neural networks. Here, we investigate how modeling three distinct components of mammalian sleep together affects continual learning in artificial neural networks.
  arXiv  Detail & Related papers  (2022-09-09T13:45:27Z)
• Memory-enriched computation and learning in spiking neural networks through Hebbian plasticity [9.453554184019108]
  Hebbian plasticity is believed to play a pivotal role in biological memory. We introduce a novel spiking neural network architecture that is enriched by Hebbian synaptic plasticity. We show that Hebbian enrichment renders spiking neural networks surprisingly versatile in terms of their computational as well as learning capabilities.
  arXiv  Detail & Related papers  (2022-05-23T12:48:37Z)
• Replay in Deep Learning: Current Approaches and Missing Biological Elements [33.20770284464084]
  Replay is the reactivation of one or more neural patterns. It is thought to play a critical role in memory formation, retrieval, and consolidation. We provide the first comprehensive comparison between replay in the mammalian brain and replay in artificial neural networks.
  arXiv  Detail & Related papers  (2021-04-01T15:19:08Z)
• Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting [135.0863818867184]
  artificial neural variability (ANV) helps artificial neural networks learn some advantages from natural'' neural networks. ANV plays as an implicit regularizer of the mutual information between the training data and the learned model. It can effectively relieve overfitting, label noise memorization, and catastrophic forgetting at negligible costs.
  arXiv  Detail & Related papers  (2020-11-12T06:06:33Z)
• Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes [0.0]
  Continual algorithms are designed to accumulate and improve knowledge in a curriculum of learning-experiences without forgetting. Generative Replay consists of regenerating past learning experiences with a generative model to remember them. We show that they are very promising methods for continual learning.
  arXiv  Detail & Related papers  (2020-07-01T13:44:33Z)
• A Developmental Neuro-Robotics Approach for Boosting the Recognition of Handwritten Digits [91.3755431537592]
  Recent evidence shows that a simulation of the children's embodied strategies can improve the machine intelligence too. This article explores the application of embodied strategies to convolutional neural network models in the context of developmental neuro-robotics.
  arXiv  Detail & Related papers  (2020-03-23T14:55:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.