Incremental Evolution and Development of Deep Artificial Neural Networks

• URL: http://arxiv.org/abs/2004.00302v1
• Date: Wed, 1 Apr 2020 09:23:15 GMT
• Title: Incremental Evolution and Development of Deep Artificial Neural Networks
• Abstract summary: We extend Fast Deep Evolutionary Network Structured Representation (Fast-DENSER) to incremental development. Results show that the average performance of the models generated by incremental development is statistically superior to the non-incremental average performance.
• Score: 1.6631602844999722
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: NeuroEvolution (NE) methods are known for applying Evolutionary Computation to the optimisation of Artificial Neural Networks(ANNs). Despite aiding non-expert users to design and train ANNs, the vast majority of NE approaches disregard the knowledge that is gathered when solving other tasks, i.e., evolution starts from scratch for each problem, ultimately delaying the evolutionary process. To overcome this drawback, we extend Fast Deep Evolutionary Network Structured Representation (Fast-DENSER) to incremental development. We hypothesise that by transferring the knowledge gained from previous tasks we can attain superior results and speedup evolution. The results show that the average performance of the models generated by incremental development is statistically superior to the non-incremental average performance. In case the number of evaluations performed by incremental development is smaller than the performed by non-incremental development the attained results are similar in performance, which indicates that incremental development speeds up evolution. Lastly, the models generated using incremental development generalise better, and thus, without further evolution, report a superior performance on unseen problems.

Related papers

• Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning [51.17262419591891]
  Test-time scaling through iterative self-evolution with environment feedback, as demonstrated by AlphaEvolve, shows remarkable performance gains.<n>We present MetaEvolve, a framework designed to develop these meta-skills via a data synthesis pipeline, evolution-aware reinforcement learning, and inference-time evolutionary search.
  arXiv  Detail & Related papers  (2026-07-24T04:35:29Z)
• Evolutionary Discovery of Developmental Reward Schedules in Deep Reinforcement Learning [0.0]
  We propose an evolutionary framework for discovering developmental reward schedules.<n>Agency, novelty, and reactivity are combined through time-varying weights.<n>Our results position evolutionary optimization as a promising approach for developmental reward schedule discovery.
  arXiv  Detail & Related papers  (2026-06-18T18:47:26Z)
• Squeeze Evolve: Unified Multi-Model Orchestration for Verifier-Free Evolution [81.46210789228296]
  We introduce Squeeze Evolve, a unified multi-model orchestration framework for verifier-free evolutionary inference.<n>Our approach is guided by a simple principle: allocate model capability where it has the highest marginal utility.
  arXiv  Detail & Related papers  (2026-04-09T02:14:45Z)
• CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpad [72.197582948888]
  We develop CausalEvolve, equipped with a causal scratchpad that leverages Large Language Models (LLMs) to identify and reason about guiding factors for evolution.<n>CaulEvolve first identifies outcome-level factors that offer complementary inspirations in improving the target objective.<n>During the evolution, CausalEvolve also inspects surprise patterns during the evolution and abductive reasoning to hypothesize new factors, which in turn offer novel directions.
  arXiv  Detail & Related papers  (2026-03-15T19:53:31Z)
• DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution [28.737322041874293]
  LLM-driven evolutionary systems have shown promise for automated science discovery.<n>Existing approaches such as AlphaEvolve rely on full-code histories that are context-inefficient.<n>We propose DeltaEvolve, a momentum-driven evolutionary framework that replaces full-code history with structured semantic delta.
  arXiv  Detail & Related papers  (2026-02-02T23:47:54Z)
• Position: Agentic Evolution is the Path to Evolving LLMs [56.733933092220845]
  We argue that addressing this limitation requires a new scaling axis-evolution.<n>Existing deployment-time adaptation methods lack the strategic agency needed to diagnose failures and produce durable improvements.
  arXiv  Detail & Related papers  (2026-01-30T22:15:58Z)
• General Self-Prediction Enhancement for Spiking Neurons [71.01912385372577]
  Spiking Neural Networks (SNNs) are highly energy-efficient due to event-driven, sparse computation, but their training is challenged by spike non-differentiability and trade-offs among performance, efficiency, and biological plausibility.<n>We propose a self-prediction enhanced spiking neuron method that generates an internal prediction current from its input-output history to modulate membrane potential.<n>This design offers dual advantages, it creates a continuous gradient path that alleviates vanishing gradients and boosts training stability and accuracy, while also aligning with biological principles, which resembles distal dendritic modulation and error-driven synaptic plasticity.
  arXiv  Detail & Related papers  (2026-01-29T15:08:48Z)
• Directed evolution algorithm drives neural prediction [6.481103165281994]
  directed evolution model (DEM) is a novel computational model that mimics the trial-and-error processes of biological directed evolution.<n>DEM can efficiently improve the performance of cross-domain pre-implantation neural predictions while addressing the challenge of label scarcity in target domain.
  arXiv  Detail & Related papers  (2025-12-01T07:17:45Z)
• ThetaEvolve: Test-time Learning on Open Problems [110.5756538358217]
  We introduce ThetaEvolve, an open-source framework that simplifies and extends AlphaEvolve to efficiently scale both in-context learning and Reinforcement Learning (RL) at test time.<n>We find that ThetaEvolve with RL at test-time consistently outperforms inference-only baselines.
  arXiv  Detail & Related papers  (2025-11-28T18:58:14Z)
• Evolution imposes an inductive bias that alters and accelerates learning dynamics [49.1574468325115]
  We investigate the effect of evolutionary optimization on the learning dynamics of neural networks.<n>We combined algorithms natural selection and online learning to produce a method for evolutionarily conditioning artificial neural networks.<n>Results suggest evolution constitutes an inductive bias that tunes neural systems to enable rapid learning.
  arXiv  Detail & Related papers  (2025-05-15T18:50:57Z)
• Low Rank Factorizations are Indirect Encodings for Deep Neuroevolution [0.0]
  We introduce low-rank, factorized neuroevolution: an indirect encoding through which we can search a small space of low-rank factors. We evaluate our method on a language modeling task using transformers, as well as continuous and discrete vision-based reinforcement learning tasks.
  arXiv  Detail & Related papers  (2025-04-03T21:31:48Z)
• Impacts of Darwinian Evolution on Pre-trained Deep Neural Networks [12.847052918757768]
  This work proposes a neural network optimization framework based on evolutionary theory. Experiments are carried out to examine the relationships between Darwinian evolution and neural network optimization. The proposed framework has positive impacts on the network, with reduced over-fitting and an order of magnitude lower time complexity.
  arXiv  Detail & Related papers  (2024-08-10T14:08:33Z)
• Trackable Agent-based Evolution Models at Wafer Scale [0.0]
  We focus on the problem of extracting phylogenetic information from agent-based evolution on the 850,000 processor Cerebras Wafer Scale Engine (WSE) We present an asynchronous island-based genetic algorithm (GA) framework for WSE hardware. We validate phylogenetic reconstructions from these trials and demonstrate their suitability for inference of underlying evolutionary conditions.
  arXiv  Detail & Related papers  (2024-04-16T19:24:14Z)
• NeuroLGP-SM: A Surrogate-assisted Neuroevolution Approach using Linear Genetic Programming [0.19116784879310028]
  We propose a new approach to training deep neural networks (DNNs) called NeuroLGP-Surrogate Model (NeuroLGP-SM)
  arXiv  Detail & Related papers  (2024-03-28T14:31:01Z)
• DARLEI: Deep Accelerated Reinforcement Learning with Evolutionary Intelligence [77.78795329701367]
  We present DARLEI, a framework that combines evolutionary algorithms with parallelized reinforcement learning. We characterize DARLEI's performance under various conditions, revealing factors impacting diversity of evolved morphologies. We hope to extend DARLEI in future work to include interactions between diverse morphologies in richer environments.
  arXiv  Detail & Related papers  (2023-12-08T16:51:10Z)
• Solving Large-scale Spatial Problems with Convolutional Neural Networks [88.31876586547848]
  We employ transfer learning to improve training efficiency for large-scale spatial problems. We propose that a convolutional neural network (CNN) can be trained on small windows of signals, but evaluated on arbitrarily large signals with little to no performance degradation.
  arXiv  Detail & Related papers  (2023-06-14T01:24:42Z)
• Multiobjective Evolutionary Pruning of Deep Neural Networks with Transfer Learning for improving their Performance and Robustness [15.29595828816055]
  This work proposes MO-EvoPruneDeepTL, a multi-objective evolutionary pruning algorithm. We use Transfer Learning to adapt the last layers of Deep Neural Networks, by replacing them with sparse layers evolved by a genetic algorithm. Experiments show that our proposal achieves promising results in all the objectives, and direct relation are presented.
  arXiv  Detail & Related papers  (2023-02-20T19:33:38Z)
• Epigenetic evolution of deep convolutional models [81.21462458089142]
  We build upon a previously proposed neuroevolution framework to evolve deep convolutional models. We propose a convolutional layer layout which allows kernels of different shapes and sizes to coexist within the same layer. The proposed layout enables the size and shape of individual kernels within a convolutional layer to be evolved with a corresponding new mutation operator.
  arXiv  Detail & Related papers  (2021-04-12T12:45:16Z)
• EvoPose2D: Pushing the Boundaries of 2D Human Pose Estimation using Accelerated Neuroevolution with Weight Transfer [82.28607779710066]
  We explore the application of neuroevolution, a form of neural architecture search inspired by biological evolution, in the design of 2D human pose networks. Our method produces network designs that are more efficient and more accurate than state-of-the-art hand-designed networks.
  arXiv  Detail & Related papers  (2020-11-17T05:56:16Z)
• Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning [56.83172249278467]
  We introduce Evolutionary Graph Reinforcement Learning (EGRL), a method designed for large search spaces. We train and validate our approach directly on the Intel NNP-I chip for inference. We additionally achieve 28-78% speed-up compared to the native NNP-I compiler on all three workloads.
  arXiv  Detail & Related papers  (2020-07-14T18:50:12Z)
• Sampled Training and Node Inheritance for Fast Evolutionary Neural Architecture Search [22.483917379706725]
  evolutionary neural architecture search (ENAS) has received increasing attention due to the attractive global optimization capability of evolutionary algorithms. This paper proposes a new framework for fast ENAS based on directed acyclic graph, in which parents are randomly sampled and trained on each mini-batch of training data. We evaluate the proposed algorithm on the widely used datasets, in comparison with 26 state-of-the-art peer algorithms.
  arXiv  Detail & Related papers  (2020-03-07T12:33:01Z)
• Alpha Discovery Neural Network based on Prior Knowledge [55.65102700986668]
  Genetic programming (GP) is the state-of-the-art in financial automated feature construction task. This paper proposes Alpha Discovery Neural Network (ADNN), a tailored neural network structure which can automatically construct diversified financial technical indicators.
  arXiv  Detail & Related papers  (2019-12-26T03:10:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.