Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges

• URL: http://arxiv.org/abs/2006.05415v1
• Date: Tue, 9 Jun 2020 17:28:25 GMT
• Title: Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges
• Authors: Edgar Galv\'an and Peter Mooney
• Abstract summary: A variety of methods have been applied to the architectural configuration and learning or training of artificial deep neural networks (DNN) Evolutionary Algorithms (EAs) are gaining momentum as a computationally feasible method for the automated optimisation and training of DNNs. This paper presents a comprehensive survey, discussion and evaluation of the state-of-the-art works on using EAs for architectural configuration and training of DNNs.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A variety of methods have been applied to the architectural configuration and learning or training of artificial deep neural networks (DNN). These methods play a crucial role in the success or failure of the DNN for most problems and applications. Evolutionary Algorithms (EAs) are gaining momentum as a computationally feasible method for the automated optimisation and training of DNNs. Neuroevolution is a term which describes these processes of automated configuration and training of DNNs using EAs. While many works exist in the literature, no comprehensive surveys currently exist focusing exclusively on the strengths and limitations of using neuroevolution approaches in DNNs. Prolonged absence of such surveys can lead to a disjointed and fragmented field preventing DNNs researchers potentially adopting neuroevolutionary methods in their own research, resulting in lost opportunities for improving performance and wider application within real-world deep learning problems. This paper presents a comprehensive survey, discussion and evaluation of the state-of-the-art works on using EAs for architectural configuration and training of DNNs. Based on this survey, the paper highlights the most pertinent current issues and challenges in neuroevolution and identifies multiple promising future research directions.

Related papers

• Enhancing learning in spiking neural networks through neuronal heterogeneity and neuromodulatory signaling [52.06722364186432]
  We propose a biologically-informed framework for enhancing artificial neural networks (ANNs) Our proposed dual-framework approach highlights the potential of spiking neural networks (SNNs) for emulating diverse spiking behaviors. We outline how the proposed approach integrates brain-inspired compartmental models and task-driven SNNs, bioinspiration and complexity.
  arXiv  Detail & Related papers  (2024-07-05T14:11:28Z)
• Evolutionary Spiking Neural Networks: A Survey [21.27024959576232]
  Spiking neural networks (SNNs) are gaining increasing attention as potential computationally efficient alternatives to traditional artificial neural networks(ANNs) The unique information propagation mechanisms and the complexity of SNN neuron models pose challenges for adopting traditional methods developed for ANNs to SNNs. Recently, a novel paradigm utilizing evolutionary computation methods has emerged to tackle these challenges.
  arXiv  Detail & Related papers  (2024-06-18T12:27:55Z)
• Direct Training High-Performance Deep Spiking Neural Networks: A Review of Theories and Methods [33.377770671553336]
  Spiking neural networks (SNNs) offer a promising energy-efficient alternative to artificial neural networks (ANNs) In this paper, we provide a new perspective to summarize the theories and methods for training deep SNNs with high performance.
  arXiv  Detail & Related papers  (2024-05-06T09:58:54Z)
• Brain-Inspired Spiking Neural Networks for Industrial Fault Diagnosis: A Survey, Challenges, and Opportunities [10.371337760495521]
  Spiking Neural Network (SNN) is founded on principles of Brain-inspired computing. This paper systematically reviews the theoretical progress of SNN-based models to answer the question of what SNN is.
  arXiv  Detail & Related papers  (2023-11-13T11:25:34Z)
• Direct Learning-Based Deep Spiking Neural Networks: A Review [17.255056657521195]
  spiking neural network (SNN) is a promising brain-inspired computational model with binary spike information transmission mechanism. In this paper, we present a survey of direct learning-based deep SNN works, mainly categorized into accuracy improvement methods, efficiency improvement methods, and temporal dynamics utilization methods.
  arXiv  Detail & Related papers  (2023-05-31T10:32:16Z)
• A Hybrid Neural Coding Approach for Pattern Recognition with Spiking Neural Networks [53.31941519245432]
  Brain-inspired spiking neural networks (SNNs) have demonstrated promising capabilities in solving pattern recognition tasks. These SNNs are grounded on homogeneous neurons that utilize a uniform neural coding for information representation. In this study, we argue that SNN architectures should be holistically designed to incorporate heterogeneous coding schemes.
  arXiv  Detail & Related papers  (2023-05-26T02:52:12Z)
• Neuroevolutionary Multi-objective approaches to Trajectory Prediction in Autonomous Vehicles [2.9552300389898094]
  We focus on the intersection of neuroevolution and evolutionary multi-objective optimization. We study a rich convolutional neural network composed of a CNN and Long-short Term Memory network. We show how these objectives have either a positive or detrimental effect in neuroevolution for trajectory prediction in autonomous vehicles.
  arXiv  Detail & Related papers  (2022-05-04T15:03:26Z)
• Exploiting Heterogeneity in Operational Neural Networks by Synaptic Plasticity [87.32169414230822]
  Recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) In this study the focus is drawn on searching the best-possible operator set(s) for the hidden neurons of the network based on the Synaptic Plasticity paradigm that poses the essential learning theory in biological neurons. Experimental results over highly challenging problems demonstrate that the elite ONNs even with few neurons and layers can achieve a superior learning performance than GIS-based ONNs.
  arXiv  Detail & Related papers  (2020-08-21T19:03:23Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)
• Boosting Deep Neural Networks with Geometrical Prior Knowledge: A Survey [77.99182201815763]
  Deep Neural Networks (DNNs) achieve state-of-the-art results in many different problem settings. DNNs are often treated as black box systems, which complicates their evaluation and validation. One promising field, inspired by the success of convolutional neural networks (CNNs) in computer vision tasks, is to incorporate knowledge about symmetric geometrical transformations.
  arXiv  Detail & Related papers  (2020-06-30T14:56:05Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.