V1T: large-scale mouse V1 response prediction using a Vision Transformer

• URL: http://arxiv.org/abs/2302.03023v4
• Date: Tue, 5 Sep 2023 17:56:42 GMT
• Title: V1T: large-scale mouse V1 response prediction using a Vision Transformer
• Authors: Bryan M. Li, Isabel M. Cornacchia, Nathalie L. Rochefort, Arno Onken
• Abstract summary: We introduce V1T, a novel Vision Transformer based architecture that learns a shared visual and behavioral representation across animals. We evaluate our model on two large datasets recorded from mouse primary visual cortex and outperform previous convolution-based models by more than 12.7% in prediction performance.
• Score: 1.5703073293718952
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate predictive models of the visual cortex neural response to natural visual stimuli remain a challenge in computational neuroscience. In this work, we introduce V1T, a novel Vision Transformer based architecture that learns a shared visual and behavioral representation across animals. We evaluate our model on two large datasets recorded from mouse primary visual cortex and outperform previous convolution-based models by more than 12.7% in prediction performance. Moreover, we show that the self-attention weights learned by the Transformer correlate with the population receptive fields. Our model thus sets a new benchmark for neural response prediction and can be used jointly with behavioral and neural recordings to reveal meaningful characteristic features of the visual cortex.

Related papers

• Matching the Neuronal Representations of V1 is Necessary to Improve Robustness in CNNs with V1-like Front-ends [1.8434042562191815]
  Recently, it was shown that simulating computations in early visual areas at the front of convolutional neural networks leads to improvements in robustness to image corruptions. Here, we show that the neuronal representations that emerge from precisely matching the distribution of RF properties found in primate V1 is key for this improvement in robustness.
  arXiv  Detail & Related papers  (2023-10-16T16:52:15Z)
• Brain-like representational straightening of natural movies in robust feedforward neural networks [2.8749107965043286]
  Representational straightening refers to a decrease in curvature of visual feature representations of a sequence of frames taken from natural movies. We show robustness to noise in the input image can produce representational straightening in feedforward neural networks.
  arXiv  Detail & Related papers  (2023-08-26T13:04:36Z)
• Adapting Brain-Like Neural Networks for Modeling Cortical Visual Prostheses [68.96380145211093]
  Cortical prostheses are devices implanted in the visual cortex that attempt to restore lost vision by electrically stimulating neurons. Currently, the vision provided by these devices is limited, and accurately predicting the visual percepts resulting from stimulation is an open challenge. We propose to address this challenge by utilizing 'brain-like' convolutional neural networks (CNNs), which have emerged as promising models of the visual system.
  arXiv  Detail & Related papers  (2022-09-27T17:33:19Z)
• The Sensorium competition on predicting large-scale mouse primary visual cortex activity [28.272130531998936]
  We propose the Sensorium benchmark competition to identify state-of-the-art models of the mouse visual system. We collected a large-scale dataset from mouse primary visual cortex containing the responses of more than 28,000 neurons. The benchmark challenge will rank models based on predictive performance for neuronal responses on a held-out test set.
  arXiv  Detail & Related papers  (2022-06-17T10:09:57Z)
• Top-down inference in an early visual cortex inspired hierarchical Variational Autoencoder [0.0]
  We exploit advances in Variational Autoencoders to investigate the early visual cortex with sparse coding hierarchical VAEs trained on natural images. We show that representations similar to the one found in the primary and secondary visual cortices naturally emerge under mild inductive biases. We show that a neuroscience-inspired choice of the recognition model is critical for two signatures of computations with generative models.
  arXiv  Detail & Related papers  (2022-06-01T12:21:58Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Self-Supervised Graph Representation Learning for Neuronal Morphologies [75.38832711445421]
  We present GraphDINO, a data-driven approach to learn low-dimensional representations of 3D neuronal morphologies from unlabeled datasets. We show, in two different species and across multiple brain areas, that this method yields morphological cell type clusterings on par with manual feature-based classification by experts. Our method could potentially enable data-driven discovery of novel morphological features and cell types in large-scale datasets.
  arXiv  Detail & Related papers  (2021-12-23T12:17:47Z)
• STAR: Sparse Transformer-based Action Recognition [61.490243467748314]
  This work proposes a novel skeleton-based human action recognition model with sparse attention on the spatial dimension and segmented linear attention on the temporal dimension of data. Experiments show that our model can achieve comparable performance while utilizing much less trainable parameters and achieve high speed in training and inference.
  arXiv  Detail & Related papers  (2021-07-15T02:53:11Z)
• Deep Reinforcement Learning Models Predict Visual Responses in the Brain: A Preliminary Result [1.0323063834827415]
  We use reinforcement learning to train neural network models to play a 3D computer game. We find that these reinforcement learning models achieve neural response prediction accuracy scores in the early visual areas. In contrast, the supervised neural network models yield better neural response predictions in the higher visual areas.
  arXiv  Detail & Related papers  (2021-06-18T13:10:06Z)
• The Neural Coding Framework for Learning Generative Models [91.0357317238509]
  We propose a novel neural generative model inspired by the theory of predictive processing in the brain. In a similar way, artificial neurons in our generative model predict what neighboring neurons will do, and adjust their parameters based on how well the predictions matched reality.
  arXiv  Detail & Related papers  (2020-12-07T01:20:38Z)
• Muti-view Mouse Social Behaviour Recognition with Deep Graphical Model [124.26611454540813]
  Social behaviour analysis of mice is an invaluable tool to assess therapeutic efficacy of neurodegenerative diseases. Because of the potential to create rich descriptions of mouse social behaviors, the use of multi-view video recordings for rodent observations is increasingly receiving much attention. We propose a novel multiview latent-attention and dynamic discriminative model that jointly learns view-specific and view-shared sub-structures.
  arXiv  Detail & Related papers  (2020-11-04T18:09:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.