Episodic memory governs choices: An RNN-based reinforcement learning model for decision-making task

• URL: http://arxiv.org/abs/2103.03679v1
• Date: Sun, 24 Jan 2021 04:33:07 GMT
• Title: Episodic memory governs choices: An RNN-based reinforcement learning model for decision-making task
• Authors: Xiaohan Zhang, Lu Liu, Guodong Long, Jing Jiang, Shenquan Liu
• Abstract summary: We develop an RNN-based Actor-Critic framework to solve two tasks analogous to the monkeys' decision-making tasks. We try to explore an open question in neuroscience: which episodic memory in the hippocampus should be selected to ultimately govern future decisions.
• Score: 24.96447960548042
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Typical methods to study cognitive function are to record the electrical activities of animal neurons during the training of animals performing behavioral tasks. A key problem is that they fail to record all the relevant neurons in the animal brain. To alleviate this problem, we develop an RNN-based Actor-Critic framework, which is trained through reinforcement learning (RL) to solve two tasks analogous to the monkeys' decision-making tasks. The trained model is capable of reproducing some features of neural activities recorded from animal brain, or some behavior properties exhibited in animal experiments, suggesting that it can serve as a computational platform to explore other cognitive functions. Furthermore, we conduct behavioral experiments on our framework, trying to explore an open question in neuroscience: which episodic memory in the hippocampus should be selected to ultimately govern future decisions. We find that the retrieval of salient events sampled from episodic memories can effectively shorten deliberation time than common events in the decision-making process. The results indicate that salient events stored in the hippocampus could be prioritized to propagate reward information, and thus allow decision-makers to learn a strategy faster.

Related papers

• Towards a "universal translator" for neural dynamics at single-cell, single-spike resolution [10.49121904052395]
  We build towards a first foundation model for neural spiking data that can solve a diverse set of tasks across multiple brain areas. Prediction tasks include single-neuron and region-level activity prediction, forward prediction, and behavior decoding.
  arXiv  Detail & Related papers  (2024-07-19T21:05:28Z)
• Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks [74.3099028063756]
  We develop a new method with neuronal operations based on lateral connections and Hebbian learning. We show that Hebbian and anti-Hebbian learning on recurrent lateral connections can effectively extract the principal subspace of neural activities. Our method consistently solves for spiking neural networks with nearly zero forgetting.
  arXiv  Detail & Related papers  (2024-02-19T09:29:37Z)
• Brain-Like Replay Naturally Emerges in Reinforcement Learning Agents [4.603243771244471]
  We discover naturally emergent replay under task-optimized paradigm using a recurrent neural network-based reinforcement learning model. Our work provides a new avenue for understanding the mechanisms behind replay.
  arXiv  Detail & Related papers  (2024-02-02T14:55:51Z)
• A Goal-Driven Approach to Systems Neuroscience [2.6451153531057985]
  Humans and animals exhibit a range of interesting behaviors in dynamic environments. It is unclear how our brains actively reformat this dense sensory information to enable these behaviors. We offer a new definition of interpretability that we show has promise in yielding unified structural and functional models of neural circuits.
  arXiv  Detail & Related papers  (2023-11-05T16:37:53Z)
• Memory-Augmented Theory of Mind Network [59.9781556714202]
  Social reasoning requires the capacity of theory of mind (ToM) to contextualise and attribute mental states to others. Recent machine learning approaches to ToM have demonstrated that we can train the observer to read the past and present behaviours of other agents. We tackle the challenges by equipping the observer with novel neural memory mechanisms to encode, and hierarchical attention to selectively retrieve information about others. This results in ToMMY, a theory of mind model that learns to reason while making little assumptions about the underlying mental processes.
  arXiv  Detail & Related papers  (2023-01-17T14:48:58Z)
• Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps [75.84770193489639]
  We examine the impact of test loss, training corpus and model architecture on the prediction of functional Magnetic Resonance Imaging timecourses of participants listening to an audiobook. We find that untrained versions of each model already explain significant amount of signal in the brain by capturing similarity in brain responses across identical words. We suggest good practices for future studies aiming at explaining the human language system using neural language models.
  arXiv  Detail & Related papers  (2022-07-07T15:37:17Z)
• A bio-inspired implementation of a sparse-learning spike-based hippocampus memory model [0.0]
  We propose a novel bio-inspired memory model based on the hippocampus. It can learn memories, recall them from a cue and even forget memories when trying to learn others with the same cue. This work presents the first hardware implementation of a fully functional bio-inspired spike-based hippocampus memory model.
  arXiv  Detail & Related papers  (2022-06-10T07:48:29Z)
• Overcoming the Domain Gap in Neural Action Representations [60.47807856873544]
  3D pose data can now be reliably extracted from multi-view video sequences without manual intervention. We propose to use it to guide the encoding of neural action representations together with a set of neural and behavioral augmentations. To reduce the domain gap, during training, we swap neural and behavioral data across animals that seem to be performing similar actions.
  arXiv  Detail & Related papers  (2021-12-02T12:45:46Z)
• Overcoming the Domain Gap in Contrastive Learning of Neural Action Representations [60.47807856873544]
  A fundamental goal in neuroscience is to understand the relationship between neural activity and behavior. We generated a new multimodal dataset consisting of the spontaneous behaviors generated by fruit flies. This dataset and our new set of augmentations promise to accelerate the application of self-supervised learning methods in neuroscience.
  arXiv  Detail & Related papers  (2021-11-29T15:27:51Z)
• Neuronal Learning Analysis using Cycle-Consistent Adversarial Networks [4.874780144224057]
  We use a variant of deep generative models called - CycleGAN, to learn the unknown mapping between pre- and post-learning neural activities. We develop an end-to-end pipeline to preprocess, train and evaluate calcium fluorescence signals, and a procedure to interpret the resulting deep learning models.
  arXiv  Detail & Related papers  (2021-11-25T13:24:19Z)
• Neuroevolution of a Recurrent Neural Network for Spatial and Working Memory in a Simulated Robotic Environment [57.91534223695695]
  We evolved weights in a biologically plausible recurrent neural network (RNN) using an evolutionary algorithm to replicate the behavior and neural activity observed in rats. Our method demonstrates how the dynamic activity in evolved RNNs can capture interesting and complex cognitive behavior.
  arXiv  Detail & Related papers  (2021-02-25T02:13:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.