Developing Convolutional Neural Networks using a Novel Lamarckian Co-Evolutionary Algorithm

• URL: http://arxiv.org/abs/2410.22487v1
• Date: Tue, 29 Oct 2024 19:26:23 GMT
• Title: Developing Convolutional Neural Networks using a Novel Lamarckian Co-Evolutionary Algorithm
• Authors: Zaniar Sharifi, Khabat Soltanian, Ali Amiri,
• Abstract summary: This paper introduces LCoDeepNEAT, an instantiation of Lamarckian genetic algorithms. LCoDeepNEAT co-evolves CNN architectures and their respective final layer weights. Our method yields a notable improvement in the classification accuracy of candidate solutions, ranging from 2% to 5.6%.
• Score: 1.2425910171551517
• License:
• Abstract: Neural Architecture Search (NAS) methods autonomously discover high-accuracy neural network architectures, outperforming manually crafted ones. However, The NAS methods require high computational costs due to the high dimension search space and the need to train multiple candidate solutions. This paper introduces LCoDeepNEAT, an instantiation of Lamarckian genetic algorithms, which extends the foundational principles of the CoDeepNEAT framework. LCoDeepNEAT co-evolves CNN architectures and their respective final layer weights. The evaluation process of LCoDeepNEAT entails a single epoch of SGD, followed by the transference of the acquired final layer weights to the genetic representation of the network. In addition, it expedites the process of evolving by imposing restrictions on the architecture search space, specifically targeting architectures comprising just two fully connected layers for classification. Our method yields a notable improvement in the classification accuracy of candidate solutions throughout the evolutionary process, ranging from 2% to 5.6%. This outcome underscores the efficacy and effectiveness of integrating gradient information and evolving the last layer of candidate solutions within LCoDeepNEAT. LCoDeepNEAT is assessed across six standard image classification datasets and benchmarked against eight leading NAS methods. Results demonstrate LCoDeepNEAT's ability to swiftly discover competitive CNN architectures with fewer parameters, conserving computational resources, and achieving superior classification accuracy compared to other approaches.

Related papers

• Informed deep hierarchical classification: a non-standard analysis inspired approach [0.0]
  It consists in a multi-output deep neural network equipped with specific projection operators placed before each output layer. The design of such an architecture, called lexicographic hybrid deep neural network (LH-DNN), has been possible by combining tools from different and quite distant research fields. To assess the efficacy of the approach, the resulting network is compared against the B-CNN, a convolutional neural network tailored for hierarchical classification tasks.
  arXiv  Detail & Related papers  (2024-09-25T14:12:50Z)
• SA-CNN: Application to text categorization issues using simulated annealing-based convolutional neural network optimization [0.0]
  Convolutional neural networks (CNNs) are a representative class of deep learning algorithms. We introduce SA-CNN neural networks for text classification tasks based on Text-CNN neural networks.
  arXiv  Detail & Related papers  (2023-03-13T14:27:34Z)
• Optimization-Based Separations for Neural Networks [57.875347246373956]
  We show that gradient descent can efficiently learn ball indicator functions using a depth 2 neural network with two layers of sigmoidal activations. This is the first optimization-based separation result where the approximation benefits of the stronger architecture provably manifest in practice.
  arXiv  Detail & Related papers  (2021-12-04T18:07:47Z)
• Non-Gradient Manifold Neural Network [79.44066256794187]
  Deep neural network (DNN) generally takes thousands of iterations to optimize via gradient descent. We propose a novel manifold neural network based on non-gradient optimization.
  arXiv  Detail & Related papers  (2021-06-15T06:39:13Z)
• Learning Structures for Deep Neural Networks [99.8331363309895]
  We propose to adopt the efficient coding principle, rooted in information theory and developed in computational neuroscience. We show that sparse coding can effectively maximize the entropy of the output signals. Our experiments on a public image classification dataset demonstrate that using the structure learned from scratch by our proposed algorithm, one can achieve a classification accuracy comparable to the best expert-designed structure.
  arXiv  Detail & Related papers  (2021-05-27T12:27:24Z)
• A SAR speckle filter based on Residual Convolutional Neural Networks [68.8204255655161]
  This work aims to present a novel method for filtering the speckle noise from Sentinel-1 data by applying Deep Learning (DL) algorithms, based on Convolutional Neural Networks (CNNs) The obtained results, if compared with the state of the art, show a clear improvement in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM)
  arXiv  Detail & Related papers  (2021-04-19T14:43:07Z)
• Differentiable Neural Architecture Learning for Efficient Neural Network Design [31.23038136038325]
  We introduce a novel emph architecture parameterisation based on scaled sigmoid function. We then propose a general emphiable Neural Architecture Learning (DNAL) method to optimize the neural architecture without the need to evaluate candidate neural networks.
  arXiv  Detail & Related papers  (2021-03-03T02:03:08Z)
• FactorizeNet: Progressive Depth Factorization for Efficient Network Architecture Exploration Under Quantization Constraints [93.4221402881609]
  We introduce a progressive depth factorization strategy for efficient CNN architecture exploration under quantization constraints. By algorithmically increasing the granularity of depth factorization in a progressive manner, the proposed strategy enables a fine-grained, low-level analysis of layer-wise distributions. Such a progressive depth factorization strategy also enables efficient identification of the optimal depth-factorized macroarchitecture design.
  arXiv  Detail & Related papers  (2020-11-30T07:12:26Z)
• DC-NAS: Divide-and-Conquer Neural Architecture Search [108.57785531758076]
  We present a divide-and-conquer (DC) approach to effectively and efficiently search deep neural architectures. We achieve a $75.1%$ top-1 accuracy on the ImageNet dataset, which is higher than that of state-of-the-art methods using the same search space.
  arXiv  Detail & Related papers  (2020-05-29T09:02:16Z)
• Sampled Training and Node Inheritance for Fast Evolutionary Neural Architecture Search [22.483917379706725]
  evolutionary neural architecture search (ENAS) has received increasing attention due to the attractive global optimization capability of evolutionary algorithms. This paper proposes a new framework for fast ENAS based on directed acyclic graph, in which parents are randomly sampled and trained on each mini-batch of training data. We evaluate the proposed algorithm on the widely used datasets, in comparison with 26 state-of-the-art peer algorithms.
  arXiv  Detail & Related papers  (2020-03-07T12:33:01Z)
• Evolving Neural Networks through a Reverse Encoding Tree [9.235550900581764]
  This paper advances a method which incorporates a type of topological edge coding, named Reverse HANG Tree (RET), for evolving scalable neural networks efficiently. Using RET, two types of approaches -- NEAT with Binary search encoding (Bi-NEAT) and NEAT with Golden-Section search encoding (GS-NEAT) -- have been designed to solve problems in benchmark continuous learning environments.
  arXiv  Detail & Related papers  (2020-02-03T02:29:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.