Net2Brain: A Toolbox to compare artificial vision models with human brain responses

• URL: http://arxiv.org/abs/2208.09677v1
• Date: Sat, 20 Aug 2022 13:10:28 GMT
• Title: Net2Brain: A Toolbox to compare artificial vision models with human brain responses
• Authors: Domenic Bersch, Kshitij Dwivedi, Martina Vilas, Radoslaw M. Cichy, Gemma Roig
• Abstract summary: We introduce Net2Brain, a graphical and command-line user interface toolbox. It compares the representational spaces of artificial deep neural networks (DNNs) and human brain recordings. We demonstrate the functionality and advantages of Net2Brain with an example showcasing how it can be used to test hypotheses of cognitive computational neuroscience.
• Score: 11.794563225903813
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We introduce Net2Brain, a graphical and command-line user interface toolbox for comparing the representational spaces of artificial deep neural networks (DNNs) and human brain recordings. While different toolboxes facilitate only single functionalities or only focus on a small subset of supervised image classification models, Net2Brain allows the extraction of activations of more than 600 DNNs trained to perform a diverse range of vision-related tasks (e.g semantic segmentation, depth estimation, action recognition, etc.), over both image and video datasets. The toolbox computes the representational dissimilarity matrices (RDMs) over those activations and compares them to brain recordings using representational similarity analysis (RSA), weighted RSA, both in specific ROIs and with searchlight search. In addition, it is possible to add a new data set of stimuli and brain recordings to the toolbox for evaluation. We demonstrate the functionality and advantages of Net2Brain with an example showcasing how it can be used to test hypotheses of cognitive computational neuroscience.

Related papers

• Unsupervised representation learning with Hebbian synaptic and structural plasticity in brain-like feedforward neural networks [0.0]
  We introduce and evaluate a brain-like neural network model capable of unsupervised representation learning. The model was tested on a diverse set of popular machine learning benchmarks.
  arXiv  Detail & Related papers  (2024-06-07T08:32:30Z)
• BRACTIVE: A Brain Activation Approach to Human Visual Brain Learning [11.517021103782229]
  We introduce Brain Activation Network (BRACTIVE), a transformer-based approach to studying the human visual brain. The main objective of BRACTIVE is to align the visual features of subjects with corresponding brain representations via fMRI signals. Our experiments demonstrate that BRACTIVE effectively identifies person-specific regions of interest, such as face and body-selective areas.
  arXiv  Detail & Related papers  (2024-05-29T06:50:13Z)
• Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing [72.45257414889478]
  We aim to reduce human workload by predicting connectivity between over-segmented neuron pieces. We first construct a dataset, named FlyTracing, that contains millions of pairwise connections of segments expanding the whole fly brain. We propose a novel connectivity-aware contrastive learning method to generate dense volumetric EM image embedding.
  arXiv  Detail & Related papers  (2024-01-05T19:45:12Z)
• Brain Decodes Deep Nets [9.302098067235507]
  We developed a tool for visualizing and analyzing large pre-trained vision models by mapping them onto the brain. Our innovation arises from a surprising usage of brain encoding: predicting brain fMRI measurements in response to images.
  arXiv  Detail & Related papers  (2023-12-03T04:36:04Z)
• Coupling Visual Semantics of Artificial Neural Networks and Human Brain Function via Synchronized Activations [13.956089436100106]
  We propose a novel computational framework, Synchronized Activations (Sync-ACT) to couple the visual representation spaces and semantics between ANNs and BNNs. With this approach, we are able to semantically annotate the neurons in ANNs with biologically meaningful description derived from human brain imaging.
  arXiv  Detail & Related papers  (2022-06-22T03:32:17Z)
• Functional2Structural: Cross-Modality Brain Networks Representation Learning [55.24969686433101]
  Graph mining on brain networks may facilitate the discovery of novel biomarkers for clinical phenotypes and neurodegenerative diseases. We propose a novel graph learning framework, known as Deep Signed Brain Networks (DSBN), with a signed graph encoder. We validate our framework on clinical phenotype and neurodegenerative disease prediction tasks using two independent, publicly available datasets.
  arXiv  Detail & Related papers  (2022-05-06T03:45:36Z)
• Deep Reinforcement Learning Guided Graph Neural Networks for Brain Network Analysis [61.53545734991802]
  We propose a novel brain network representation framework, namely BN-GNN, which searches for the optimal GNN architecture for each brain network. Our proposed BN-GNN improves the performance of traditional GNNs on different brain network analysis tasks.
  arXiv  Detail & Related papers  (2022-03-18T07:05:27Z)
• Explainability Tools Enabling Deep Learning in Future In-Situ Real-Time Planetary Explorations [58.720142291102135]
  Deep learning (DL) has proven to be an effective machine learning and computer vision technique. Most of the Deep Neural Network (DNN) architectures are so complex that they are considered a 'black box' In this paper, we used integrated gradients to describe the attributions of each neuron to the output classes. It provides a set of explainability tools (ET) that opens the black box of a DNN so that the individual contribution of neurons to category classification can be ranked and visualized.
  arXiv  Detail & Related papers  (2022-01-15T07:10:00Z)
• Overcoming the Domain Gap in Neural Action Representations [60.47807856873544]
  3D pose data can now be reliably extracted from multi-view video sequences without manual intervention. We propose to use it to guide the encoding of neural action representations together with a set of neural and behavioral augmentations. To reduce the domain gap, during training, we swap neural and behavioral data across animals that seem to be performing similar actions.
  arXiv  Detail & Related papers  (2021-12-02T12:45:46Z)
• Interpretation of 3D CNNs for Brain MRI Data Classification [56.895060189929055]
  We extend the previous findings in gender differences from diffusion-tensor imaging on T1 brain MRI scans. We provide the voxel-wise 3D CNN interpretation comparing the results of three interpretation methods.
  arXiv  Detail & Related papers  (2020-06-20T17:56:46Z)
• On Tractable Representations of Binary Neural Networks [23.50970665150779]
  We consider the compilation of a binary neural network's decision function into tractable representations such as Ordered Binary Decision Diagrams (OBDDs) and Sentential Decision Diagrams (SDDs) In experiments, we show that it is feasible to obtain compact representations of neural networks as SDDs.
  arXiv  Detail & Related papers  (2020-04-05T03:21:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.