Dendrites endow artificial neural networks with accurate, robust and parameter-efficient learning

• URL: http://arxiv.org/abs/2404.03708v2
• Date: Fri, 13 Sep 2024 09:33:59 GMT
• Title: Dendrites endow artificial neural networks with accurate, robust and parameter-efficient learning
• Authors: Spyridon Chavlis, Panayiota Poirazi,
• Abstract summary: We show that a new ANN architecture incorporates the structured connectivity and restricted sampling properties of biological dendrites. We find that dendritic ANNs are more robust to overfitting and outperform traditional ANNs on several image classification tasks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial neural networks (ANNs) are at the core of most Deep learning (DL) algorithms that successfully tackle complex problems like image recognition, autonomous driving, and natural language processing. However, unlike biological brains who tackle similar problems in a very efficient manner, DL algorithms require a large number of trainable parameters, making them energy-intensive and prone to overfitting. Here, we show that a new ANN architecture that incorporates the structured connectivity and restricted sampling properties of biological dendrites counteracts these limitations. We find that dendritic ANNs are more robust to overfitting and outperform traditional ANNs on several image classification tasks while using significantly fewer trainable parameters. These advantages are likely the result of a different learning strategy, whereby most of the nodes in dendritic ANNs respond to multiple classes, unlike classical ANNs that strive for class-specificity. Our findings suggest that the incorporation of dendritic properties can make learning in ANNs more precise, resilient, and parameter-efficient and shed new light on how biological features can impact the learning strategies of ANNs.

Related papers

• Discovering Long-Term Effects on Parameter Efficient Fine-tuning [36.83255498301937]
  Pre-trained Artificial Neural Networks (Annns) exhibit robust pattern recognition capabilities. Annns and BNNs share extensive similarities with the human brain, specifically Biological Neural Networks (BNNs) Annns can acquire new knowledge through fine-tuning.
  arXiv  Detail & Related papers  (2024-08-24T03:27:29Z)
• Enhancing learning in spiking neural networks through neuronal heterogeneity and neuromodulatory signaling [52.06722364186432]
  We propose a biologically-informed framework for enhancing artificial neural networks (ANNs) Our proposed dual-framework approach highlights the potential of spiking neural networks (SNNs) for emulating diverse spiking behaviors. We outline how the proposed approach integrates brain-inspired compartmental models and task-driven SNNs, bioinspiration and complexity.
  arXiv  Detail & Related papers  (2024-07-05T14:11:28Z)
• A Hybrid Neural Coding Approach for Pattern Recognition with Spiking Neural Networks [53.31941519245432]
  Brain-inspired spiking neural networks (SNNs) have demonstrated promising capabilities in solving pattern recognition tasks. These SNNs are grounded on homogeneous neurons that utilize a uniform neural coding for information representation. In this study, we argue that SNN architectures should be holistically designed to incorporate heterogeneous coding schemes.
  arXiv  Detail & Related papers  (2023-05-26T02:52:12Z)
• Biologically inspired structure learning with reverse knowledge distillation for spiking neural networks [19.33517163587031]
  Spiking neural networks (SNNs) have superb characteristics in sensory information recognition tasks due to their biological plausibility. The performance of some current spiking-based models is limited by their structures which means either fully connected or too-deep structures bring too much redundancy. This paper proposes an evolutionary-based structure construction method for constructing more reasonable SNNs.
  arXiv  Detail & Related papers  (2023-04-19T08:41:17Z)
• Knowledge Enhanced Neural Networks for relational domains [83.9217787335878]
  We focus on a specific method, KENN, a Neural-Symbolic architecture that injects prior logical knowledge into a neural network. In this paper, we propose an extension of KENN for relational data.
  arXiv  Detail & Related papers  (2022-05-31T13:00:34Z)
• Motif-topology and Reward-learning improved Spiking Neural Network for Efficient Multi-sensory Integration [5.161352821775507]
  We propose a Motif-topology and Reward-learning improved spiking neural network (MR-SNN) for efficient multi-sensory integration. The experimental results showed higher accuracy and stronger robustness of the proposed MR-SNN than other conventional SNNs without using Motifs. The proposed reward learning paradigm was biologically plausible and can better explain the cognitive McGurk effect caused by incongruent visual and auditory sensory signals.
  arXiv  Detail & Related papers  (2022-02-11T02:07:44Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• Evolutionary Architecture Search for Graph Neural Networks [23.691915813153496]
  We propose a novel AutoML framework through the evolution of individual models in a large Graph Neural Networks (GNN) architecture space. To the best of our knowledge, this is the first work to introduce and evaluate evolutionary architecture search for GNN models.
  arXiv  Detail & Related papers  (2020-09-21T22:11:53Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• Towards Efficient Processing and Learning with Spikes: New Approaches for Multi-Spike Learning [59.249322621035056]
  We propose two new multi-spike learning rules which demonstrate better performance over other baselines on various tasks. In the feature detection task, we re-examine the ability of unsupervised STDP with its limitations being presented. Our proposed learning rules can reliably solve the task over a wide range of conditions without specific constraints being applied.
  arXiv  Detail & Related papers  (2020-05-02T06:41:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.