Epigenetic evolution of deep convolutional models

• URL: http://arxiv.org/abs/2104.05411v1
• Date: Mon, 12 Apr 2021 12:45:16 GMT
• Title: Epigenetic evolution of deep convolutional models
• Authors: Alexander Hadjiivanov and Alan Blair
• Abstract summary: We build upon a previously proposed neuroevolution framework to evolve deep convolutional models. We propose a convolutional layer layout which allows kernels of different shapes and sizes to coexist within the same layer. The proposed layout enables the size and shape of individual kernels within a convolutional layer to be evolved with a corresponding new mutation operator.
• Score: 81.21462458089142
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this study, we build upon a previously proposed neuroevolution framework to evolve deep convolutional models. Specifically, the genome encoding and the crossover operator are extended to make them applicable to layered networks. We also propose a convolutional layer layout which allows kernels of different shapes and sizes to coexist within the same layer, and present an argument as to why this may be beneficial. The proposed layout enables the size and shape of individual kernels within a convolutional layer to be evolved with a corresponding new mutation operator. The proposed framework employs a hybrid optimisation strategy involving structural changes through epigenetic evolution and weight update through backpropagation in a population-based setting. Experiments on several image classification benchmarks demonstrate that the crossover operator is sufficiently robust to produce increasingly performant offspring even when the parents are trained on only a small random subset of the training dataset in each epoch, thus providing direct confirmation that learned features and behaviour can be successfully transferred from parent networks to offspring in the next generation.

Related papers

• A Guide to Tracking Phylogenies in Parallel and Distributed Agent-based Evolution Models [0.0]
  In silico work with agent-based models provides an opportunity to collect high-quality records of ancestry relationships among simulated agents. Existing work generally tracks lineages directly, yielding an exact phylogenetic record of evolutionary history. Post hoc estimation is akin to how bioinformaticians build phylogenies by assessing genetic similarities between organisms.
  arXiv  Detail & Related papers  (2024-05-16T15:27:51Z)
• DARLEI: Deep Accelerated Reinforcement Learning with Evolutionary Intelligence [77.78795329701367]
  We present DARLEI, a framework that combines evolutionary algorithms with parallelized reinforcement learning. We characterize DARLEI's performance under various conditions, revealing factors impacting diversity of evolved morphologies. We hope to extend DARLEI in future work to include interactions between diverse morphologies in richer environments.
  arXiv  Detail & Related papers  (2023-12-08T16:51:10Z)
• Coevolution of Neural Architectures and Features for Stock Market Forecasting: A Multi-objective Decision Perspective [0.0]
  This paper proposes a new approach to identify a set of nondominated neural network models for further selection by the decision maker. A new coevolution approach is proposed to simultaneously select the features and topology of neural networks. The results on the NASDAQ index in pre and peri COVID time windows convincingly demonstrate that the proposed coevolution approach can evolve a set of nondominated neural forecasting models with better generalization capabilities.
  arXiv  Detail & Related papers  (2023-11-23T15:12:30Z)
• PhyloGFN: Phylogenetic inference with generative flow networks [57.104166650526416]
  We introduce the framework of generative flow networks (GFlowNets) to tackle two core problems in phylogenetics: parsimony-based and phylogenetic inference. Because GFlowNets are well-suited for sampling complex structures, they are a natural choice for exploring and sampling from the multimodal posterior distribution over tree topologies. We demonstrate that our amortized posterior sampler, PhyloGFN, produces diverse and high-quality evolutionary hypotheses on real benchmark datasets.
  arXiv  Detail & Related papers  (2023-10-12T23:46:08Z)
• Sparse Mutation Decompositions: Fine Tuning Deep Neural Networks with Subspace Evolution [0.0]
  A popular subclass of neuroevolutionary methods, called evolution strategies, relies on dense noise perturbations to mutate networks. We introduce an approach to alleviating this problem by decomposing dense mutations into low-dimensional subspaces. We conduct the first large scale exploration of neuroevolutionary fine tuning and ensembling on the notoriously difficult ImageNet dataset.
  arXiv  Detail & Related papers  (2023-02-12T01:27:26Z)
• WLD-Reg: A Data-dependent Within-layer Diversity Regularizer [98.78384185493624]
  Neural networks are composed of multiple layers arranged in a hierarchical structure jointly trained with a gradient-based optimization. We propose to complement this traditional 'between-layer' feedback with additional 'within-layer' feedback to encourage the diversity of the activations within the same layer. We present an extensive empirical study confirming that the proposed approach enhances the performance of several state-of-the-art neural network models in multiple tasks.
  arXiv  Detail & Related papers  (2023-01-03T20:57:22Z)
• EvoVGM: A Deep Variational Generative Model for Evolutionary Parameter Estimation [0.0]
  We propose a method for a deep variational Bayesian generative model that jointly approximates the true posterior of local biological evolutionary parameters. We show the consistency and effectiveness of the method on synthetic sequence alignments simulated with several evolutionary scenarios and on a real virus sequence alignment.
  arXiv  Detail & Related papers  (2022-05-25T20:08:10Z)
• X-volution: On the unification of convolution and self-attention [52.80459687846842]
  We propose a multi-branch elementary module composed of both convolution and self-attention operation. The proposed X-volution achieves highly competitive visual understanding improvements.
  arXiv  Detail & Related papers  (2021-06-04T04:32:02Z)
• Complexity-based speciation and genotype representation for neuroevolution [81.21462458089142]
  This paper introduces a speciation principle for neuroevolution where evolving networks are grouped into species based on the number of hidden neurons. The proposed speciation principle is employed in several techniques designed to promote and preserve diversity within species and in the ecosystem as a whole.
  arXiv  Detail & Related papers  (2020-10-11T06:26:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.