Encoding priors in the brain: a reinforcement learning model for mouse decision making

• URL: http://arxiv.org/abs/2112.05816v1
• Date: Fri, 10 Dec 2021 20:16:36 GMT
• Title: Encoding priors in the brain: a reinforcement learning model for mouse decision making
• Authors: Sanjukta Krishnagopal and Peter Latham
• Abstract summary: We study the International Brain Laboratory task, in which a grating appears on either the right or left side of a screen, and a mouse has to move a wheel to bring the grating to the center. We model this as a reinforcement learning task, using a feedforward neural network to map states to actions, and adjust the weights of the network to maximize reward, learning via policy gradient. Our model reproduces the main experimental finding - that the psychometric curve with respect to contrast shifts after a block switch in about 10 trials.
• Score: 1.14219428942199
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In two-alternative forced choice tasks, prior knowledge can improve performance, especially when operating near the psychophysical threshold. For instance, if subjects know that one choice is much more likely than the other, they can make that choice when evidence is weak. A common hypothesis for these kinds of tasks is that the prior is stored in neural activity. Here we propose a different hypothesis: the prior is stored in synaptic strengths. We study the International Brain Laboratory task, in which a grating appears on either the right or left side of a screen, and a mouse has to move a wheel to bring the grating to the center. The grating is often low in contrast which makes the task relatively difficult, and the prior probability that the grating appears on the right is either 80% or 20%, in (unsignaled) blocks of about 50 trials. We model this as a reinforcement learning task, using a feedforward neural network to map states to actions, and adjust the weights of the network to maximize reward, learning via policy gradient. Our model uses an internal state that stores an estimate of the grating and confidence, and follows Bayesian updates, and can switch between engaged and disengaged states to mimic animal behavior. This model reproduces the main experimental finding - that the psychometric curve with respect to contrast shifts after a block switch in about 10 trials. Also, as seen in the experiments, in our model the difference in neuronal activity in the right and left blocks is small - it is virtually impossible to decode block structure from activity on single trials if noise is about 2%. The hypothesis that priors are stored in weights is difficult to test, but the technology to do so should be available in the not so distant future.

Related papers

• Don't Cut Corners: Exact Conditions for Modularity in Biologically Inspired Representations [52.48094670415497]
  We develop a theory of when biologically inspired representations modularise with respect to source variables (sources) We derive necessary and sufficient conditions on a sample of sources that determine whether the neurons in an optimal biologically-inspired linear autoencoder modularise. Our theory applies to any dataset, extending far beyond the case of statistical independence studied in previous work.
  arXiv  Detail & Related papers  (2024-10-08T17:41:37Z)
• Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks [74.3099028063756]
  We develop a new method with neuronal operations based on lateral connections and Hebbian learning. We show that Hebbian and anti-Hebbian learning on recurrent lateral connections can effectively extract the principal subspace of neural activities. Our method consistently solves for spiking neural networks with nearly zero forgetting.
  arXiv  Detail & Related papers  (2024-02-19T09:29:37Z)
• Optimisation in Neurosymbolic Learning Systems [1.450405446885067]
  We study neurosymbolic learning, where we have both data and background knowledge expressed using symbolic languages. Probabilistic reasoning is a natural fit for neural networks, which we usually train to be probabilistic. Our insight is to train a neural network with synthetic data to predict the result of probabilistic reasoning.
  arXiv  Detail & Related papers  (2024-01-19T17:09:32Z)
• One-hot Generalized Linear Model for Switching Brain State Discovery [1.0132677989820746]
  Inferred neural interactions from neural signals primarily reflect functional interactions. We will show that the learned prior should capture the state-constant interaction, shedding light on the underlying anatomical connectome. Our methods effectively recover true interaction structures in simulated data, achieve the highest predictive likelihood with real neural datasets, and render interaction structures and hidden states more interpretable.
  arXiv  Detail & Related papers  (2023-10-23T18:10:22Z)
• Neural Bounding [12.58643866322302]
  We study the use of neural networks as bounding volumes. Our key observation is that bounding can be redefined as a problem of learning to classify space into free or occupied. We show that our neural bounding produces up to an order of magnitude fewer false positives than traditional methods.
  arXiv  Detail & Related papers  (2023-10-10T17:50:09Z)
• Benign Overfitting for Two-layer ReLU Convolutional Neural Networks [60.19739010031304]
  We establish algorithm-dependent risk bounds for learning two-layer ReLU convolutional neural networks with label-flipping noise. We show that, under mild conditions, the neural network trained by gradient descent can achieve near-zero training loss and Bayes optimal test risk.
  arXiv  Detail & Related papers  (2023-03-07T18:59:38Z)
• Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State [66.2457134675891]
  Spiking neural networks (SNNs) are brain-inspired models that enable energy-efficient implementation on neuromorphic hardware. Most existing methods imitate the backpropagation framework and feedforward architectures for artificial neural networks. We propose a novel training method that does not rely on the exact reverse of the forward computation.
  arXiv  Detail & Related papers  (2021-09-29T07:46:54Z)
• Neural Random Projection: From the Initial Task To the Input Similarity Problem [0.0]
  We propose a novel approach for implicit data representation to evaluate similarity of input data using a trained neural network. The proposed technique explicitly takes into account the initial task and significantly reduces the size of the vector representation. Our experimental results show that the proposed approach achieves competitive results on the input similarity task.
  arXiv  Detail & Related papers  (2020-10-09T13:20:24Z)
• Towards Efficient Processing and Learning with Spikes: New Approaches for Multi-Spike Learning [59.249322621035056]
  We propose two new multi-spike learning rules which demonstrate better performance over other baselines on various tasks. In the feature detection task, we re-examine the ability of unsupervised STDP with its limitations being presented. Our proposed learning rules can reliably solve the task over a wide range of conditions without specific constraints being applied.
  arXiv  Detail & Related papers  (2020-05-02T06:41:20Z)
• Non-linear Neurons with Human-like Apical Dendrite Activations [81.18416067005538]
  We show that a standard neuron followed by our novel apical dendrite activation (ADA) can learn the XOR logical function with 100% accuracy. We conduct experiments on six benchmark data sets from computer vision, signal processing and natural language processing.
  arXiv  Detail & Related papers  (2020-02-02T21:09:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.