A biologically plausible neural network for local supervision in cortical microcircuits

• URL: http://arxiv.org/abs/2011.15031v1
• Date: Mon, 30 Nov 2020 17:35:22 GMT
• Title: A biologically plausible neural network for local supervision in cortical microcircuits
• Authors: Siavash Golkar, David Lipshutz, Yanis Bahroun, Anirvan M. Sengupta, Dmitri B. Chklovskii
• Abstract summary: We derive an algorithm for training a neural network which avoids explicit error and backpropagation. Our algorithm maps onto a neural network that bears a remarkable resemblance to the connectivity structure and learning rules of the cortex.
• Score: 17.00937011213428
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The backpropagation algorithm is an invaluable tool for training artificial neural networks; however, because of a weight sharing requirement, it does not provide a plausible model of brain function. Here, in the context of a two-layer network, we derive an algorithm for training a neural network which avoids this problem by not requiring explicit error computation and backpropagation. Furthermore, our algorithm maps onto a neural network that bears a remarkable resemblance to the connectivity structure and learning rules of the cortex. We find that our algorithm empirically performs comparably to backprop on a number of datasets.

Related papers

• Coding schemes in neural networks learning classification tasks [52.22978725954347]
  We investigate fully-connected, wide neural networks learning classification tasks. We show that the networks acquire strong, data-dependent features. Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
  arXiv  Detail & Related papers  (2024-06-24T14:50:05Z)
• Verified Neural Compressed Sensing [58.98637799432153]
  We develop the first (to the best of our knowledge) provably correct neural networks for a precise computational task. We show that for modest problem dimensions (up to 50), we can train neural networks that provably recover a sparse vector from linear and binarized linear measurements. We show that the complexity of the network can be adapted to the problem difficulty and solve problems where traditional compressed sensing methods are not known to provably work.
  arXiv  Detail & Related papers  (2024-05-07T12:20:12Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Taming Binarized Neural Networks and Mixed-Integer Programs [2.7624021966289596]
  We show that binarized neural networks admit a tame representation. This makes it possible to use the framework of Bolte et al. for implicit differentiation. This approach could also be used for a broader class of mixed-integer programs.
  arXiv  Detail & Related papers  (2023-10-05T21:04:16Z)
• Addressing caveats of neural persistence with deep graph persistence [54.424983583720675]
  We find that the variance of network weights and spatial concentration of large weights are the main factors that impact neural persistence. We propose an extension of the filtration underlying neural persistence to the whole neural network instead of single layers. This yields our deep graph persistence measure, which implicitly incorporates persistent paths through the network and alleviates variance-related issues.
  arXiv  Detail & Related papers  (2023-07-20T13:34:11Z)
• Neural Bayesian Network Understudy [13.28673601999793]
  We show that a neural network can be trained to output conditional probabilities, providing approximately the same functionality as a Bayesian Network. We propose two training strategies that allow encoding the independence relations inferred from a given causal structure into the neural network.
  arXiv  Detail & Related papers  (2022-11-15T15:56:51Z)
• Seeking Interpretability and Explainability in Binary Activated Neural Networks [2.828173677501078]
  We study the use of binary activated neural networks as interpretable and explainable predictors in the context of regression tasks. We present an approach based on the efficient computation of SHAP values for quantifying the relative importance of the features, hidden neurons and even weights.
  arXiv  Detail & Related papers  (2022-09-07T20:11:17Z)
• Learning Structures for Deep Neural Networks [99.8331363309895]
  We propose to adopt the efficient coding principle, rooted in information theory and developed in computational neuroscience. We show that sparse coding can effectively maximize the entropy of the output signals. Our experiments on a public image classification dataset demonstrate that using the structure learned from scratch by our proposed algorithm, one can achieve a classification accuracy comparable to the best expert-designed structure.
  arXiv  Detail & Related papers  (2021-05-27T12:27:24Z)
• Provably Training Neural Network Classifiers under Fairness Constraints [70.64045590577318]
  We show that overparametrized neural networks could meet the constraints. Key ingredient of building a fair neural network classifier is establishing no-regret analysis for neural networks.
  arXiv  Detail & Related papers  (2020-12-30T18:46:50Z)
• On Tractable Representations of Binary Neural Networks [23.50970665150779]
  We consider the compilation of a binary neural network's decision function into tractable representations such as Ordered Binary Decision Diagrams (OBDDs) and Sentential Decision Diagrams (SDDs) In experiments, we show that it is feasible to obtain compact representations of neural networks as SDDs.
  arXiv  Detail & Related papers  (2020-04-05T03:21:26Z)
• Lossless Compression of Deep Neural Networks [17.753357839478575]
  Deep neural networks have been successful in many predictive modeling tasks, such as image and language recognition. It is challenging to deploy these networks under limited computational resources, such as in mobile devices. We introduce an algorithm that removes units and layers of a neural network while not changing the output that is produced.
  arXiv  Detail & Related papers  (2020-01-01T15:04:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.