A Study of Biologically Plausible Neural Network: The Role and
Interactions of Brain-Inspired Mechanisms in Continual Learning
- URL: http://arxiv.org/abs/2304.06738v1
- Date: Thu, 13 Apr 2023 16:34:12 GMT
- Title: A Study of Biologically Plausible Neural Network: The Role and
Interactions of Brain-Inspired Mechanisms in Continual Learning
- Authors: Fahad Sarfraz, Elahe Arani, Bahram Zonooz
- Abstract summary: Humans excel at continually acquiring, consolidating, and retaining information from an ever-changing environment, whereas artificial neural networks (ANNs) exhibit catastrophic forgetting.
We consider a biologically plausible framework that constitutes separate populations of exclusively excitatory and inhibitory neurons that adhere to Dale's principle.
We then conduct a comprehensive study on the role and interactions of different mechanisms inspired by the brain, including sparse non-overlapping representations, Hebbian learning, synaptic consolidation, and replay of past activations that accompanied the learning event.
- Score: 13.041607703862724
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Humans excel at continually acquiring, consolidating, and retaining
information from an ever-changing environment, whereas artificial neural
networks (ANNs) exhibit catastrophic forgetting. There are considerable
differences in the complexity of synapses, the processing of information, and
the learning mechanisms in biological neural networks and their artificial
counterparts, which may explain the mismatch in performance. We consider a
biologically plausible framework that constitutes separate populations of
exclusively excitatory and inhibitory neurons that adhere to Dale's principle,
and the excitatory pyramidal neurons are augmented with dendritic-like
structures for context-dependent processing of stimuli. We then conduct a
comprehensive study on the role and interactions of different mechanisms
inspired by the brain, including sparse non-overlapping representations,
Hebbian learning, synaptic consolidation, and replay of past activations that
accompanied the learning event. Our study suggests that the employing of
multiple complementary mechanisms in a biologically plausible architecture,
similar to the brain, may be effective in enabling continual learning in ANNs.
Related papers
- Enhancing learning in artificial neural networks through cellular heterogeneity and neuromodulatory signaling [52.06722364186432]
We propose a biologically-informed framework for enhancing artificial neural networks (ANNs)
Our proposed dual-framework approach highlights the potential of spiking neural networks (SNNs) for emulating diverse spiking behaviors.
We outline how the proposed approach integrates brain-inspired compartmental models and task-driven SNNs, bioinspiration and complexity.
arXiv Detail & Related papers (2024-07-05T14:11:28Z) - Exploring neural oscillations during speech perception via surrogate gradient spiking neural networks [59.38765771221084]
We present a physiologically inspired speech recognition architecture compatible and scalable with deep learning frameworks.
We show end-to-end gradient descent training leads to the emergence of neural oscillations in the central spiking neural network.
Our findings highlight the crucial inhibitory role of feedback mechanisms, such as spike frequency adaptation and recurrent connections, in regulating and synchronising neural activity to improve recognition performance.
arXiv Detail & Related papers (2024-04-22T09:40:07Z) - Brain-Inspired Machine Intelligence: A Survey of
Neurobiologically-Plausible Credit Assignment [65.268245109828]
We examine algorithms for conducting credit assignment in artificial neural networks that are inspired or motivated by neurobiology.
We organize the ever-growing set of brain-inspired learning schemes into six general families and consider these in the context of backpropagation of errors.
The results of this review are meant to encourage future developments in neuro-mimetic systems and their constituent learning processes.
arXiv Detail & Related papers (2023-12-01T05:20:57Z) - Brain-inspired learning in artificial neural networks: a review [5.064447369892274]
We review current brain-inspired learning representations in artificial neural networks.
We investigate the integration of more biologically plausible mechanisms, such as synaptic plasticity, to enhance these networks' capabilities.
arXiv Detail & Related papers (2023-05-18T18:34:29Z) - Synergistic information supports modality integration and flexible
learning in neural networks solving multiple tasks [107.8565143456161]
We investigate the information processing strategies adopted by simple artificial neural networks performing a variety of cognitive tasks.
Results show that synergy increases as neural networks learn multiple diverse tasks.
randomly turning off neurons during training through dropout increases network redundancy, corresponding to an increase in robustness.
arXiv Detail & Related papers (2022-10-06T15:36:27Z) - Overcoming the Domain Gap in Contrastive Learning of Neural Action
Representations [60.47807856873544]
A fundamental goal in neuroscience is to understand the relationship between neural activity and behavior.
We generated a new multimodal dataset consisting of the spontaneous behaviors generated by fruit flies.
This dataset and our new set of augmentations promise to accelerate the application of self-supervised learning methods in neuroscience.
arXiv Detail & Related papers (2021-11-29T15:27:51Z) - Learning by Active Forgetting for Neural Networks [36.47528616276579]
Remembering and forgetting mechanisms are two sides of the same coin in a human learning-memory system.
Modern machine learning systems have been working to endow machine with lifelong learning capability through better remembering.
This paper presents a learning model by active forgetting mechanism with artificial neural networks.
arXiv Detail & Related papers (2021-11-21T14:55:03Z) - Continuous Learning and Adaptation with Membrane Potential and
Activation Threshold Homeostasis [91.3755431537592]
This paper presents the Membrane Potential and Activation Threshold Homeostasis (MPATH) neuron model.
The model allows neurons to maintain a form of dynamic equilibrium by automatically regulating their activity when presented with input.
Experiments demonstrate the model's ability to adapt to and continually learn from its input.
arXiv Detail & Related papers (2021-04-22T04:01:32Z) - Brain-inspired self-organization with cellular neuromorphic computing
for multimodal unsupervised learning [0.0]
We propose a brain-inspired neural system based on the reentry theory using Self-Organizing Maps and Hebbian-like learning.
We show the gain of the so-called hardware plasticity induced by the ReSOM, where the system's topology is not fixed by the user but learned along the system's experience through self-organization.
arXiv Detail & Related papers (2020-04-11T21:02:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.