An information-theoretic evolutionary algorithm

• URL: http://arxiv.org/abs/2304.05963v1
• Date: Wed, 12 Apr 2023 16:36:22 GMT
• Title: An information-theoretic evolutionary algorithm
• Abstract summary: It-EA iteratively updates a search distribution with two parameters, the center, that is the bit vector at which standard bit mutation is applied, and the mutation rate. In an empirical analysis, on OneMax and LeadingOnes, the elitist and non elitist it-EAs obtain promising results.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel evolutionary algorithm on bit vectors which derives from the principles of information theory. The information-theoretic evolutionary algorithm (it-EA) iteratively updates a search distribution with two parameters, the center, that is the bit vector at which standard bit mutation is applied, and the mutation rate. The mutation rate is updated by means of information-geometric optimization and the center is updated by means of a maximum likelihood principle. Standard elitist and non elitist updates of the center are also considered. Experiments illustrate the dynamics of the mutation rate and the influence of hyperparameters. In an empirical runtime analysis, on OneMax and LeadingOnes, the elitist and non elitist it-EAs obtain promising results.

Related papers

• CliffSearch: Structured Agentic Co-Evolution over Theory and Code for Scientific Algorithm Discovery [8.600891931062277]
  We present CliffSearch, an agentic evolutionary framework in which the core evolution operators are implemented as LLM agents.<n>We illustrate the framework on three benchmark-grounded studies: transformer hyper-connection evolution, discovery on a fixed nanoGPT stack, and a smaller native-optimizer ablation.
  arXiv  Detail & Related papers  (2026-04-01T17:51:26Z)
• Reinforcement Learning Using known Invariances [54.91261509214309]
  This paper develops a theoretical framework for incorporating known group symmetries into kernel-based reinforcement learning.<n>We show that symmetry-aware RL achieves significantly better performance than their standard kernel counterparts.
  arXiv  Detail & Related papers  (2025-11-05T13:56:14Z)
• Evolution Transformer: In-Context Evolutionary Optimization [6.873777465945062]
  We introduce Evolution Transformer, a causal Transformer architecture, which can flexibly characterize a family of Evolution Strategies. We train the model weights using Evolutionary Algorithm Distillation, a technique for supervised optimization of sequence models. We analyze the resulting properties of the Evolution Transformer and propose a technique to fully self-referentially train the Evolution Transformer.
  arXiv  Detail & Related papers  (2024-03-05T14:04:13Z)
• Phylogeny-informed fitness estimation [58.720142291102135]
  We propose phylogeny-informed fitness estimation, which exploits a population's phylogeny to estimate fitness evaluations. Our results indicate that phylogeny-informed fitness estimation can mitigate the drawbacks of down-sampled lexicase. This work serves as an initial step toward improving evolutionary algorithms by exploiting runtime phylogenetic analysis.
  arXiv  Detail & Related papers  (2023-06-06T19:05:01Z)
• Optimal Algorithms for the Inhomogeneous Spiked Wigner Model [89.1371983413931]
  We derive an approximate message-passing algorithm (AMP) for the inhomogeneous problem. We identify in particular the existence of a statistical-to-computational gap where known algorithms require a signal-to-noise ratio bigger than the information-theoretic threshold to perform better than random.
  arXiv  Detail & Related papers  (2023-02-13T19:57:17Z)
• Structured mutation inspired by evolutionary theory enriches population performance and diversity [2.3577368017815705]
  Grammar-Guided Genetic Programming employs a variety of insights from evolutionary theory to autonomously design solutions for a given task. Recent insights from evolutionary biology can lead to further improvements in GGGP algorithms. We term this new method of variation Facilitated Mutation (FM)
  arXiv  Detail & Related papers  (2023-02-01T16:25:03Z)
• A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks [65.34977803841007]
  Predictive coding networks are neuroscience-inspired models with roots in both Bayesian statistics and neuroscience. We show how by simply changing the temporal scheduling of the update rule for the synaptic weights leads to an algorithm that is much more efficient and stable than the original one.
  arXiv  Detail & Related papers  (2022-11-16T00:11:04Z)
• Effective Mutation Rate Adaptation through Group Elite Selection [50.88204196504888]
  This paper introduces the Group Elite Selection of Mutation Rates (GESMR) algorithm. GESMR co-evolves a population of solutions and a population of MRs, such that each MR is assigned to a group of solutions. With the same number of function evaluations and with almost no overhead, GESMR converges faster and to better solutions than previous approaches.
  arXiv  Detail & Related papers  (2022-04-11T01:08:26Z)
• Direct Mutation and Crossover in Genetic Algorithms Applied to Reinforcement Learning Tasks [0.9137554315375919]
  This paper will focus on applying neuroevolution using a simple genetic algorithm (GA) to find the weights of a neural network that produce optimally behaving agents. We present two novel modifications that improve the data efficiency and speed of convergence when compared to the initial implementation.
  arXiv  Detail & Related papers  (2022-01-13T07:19:28Z)
• Improving RNA Secondary Structure Design using Deep Reinforcement Learning [69.63971634605797]
  We propose a new benchmark of applying reinforcement learning to RNA sequence design, in which the objective function is defined to be the free energy in the sequence's secondary structure. We show results of the ablation analysis that we do for these algorithms, as well as graphs indicating the algorithm's performance across batches.
  arXiv  Detail & Related papers  (2021-11-05T02:54:06Z)
• An Experimental Study of Weight Initialization and Weight Inheritance Effects on Neuroevolution [2.3274138116397736]
  In neuroevolution, weights typically need to be exploding at three different times: when initial genomes (ANNs) are created at the beginning of the search, when offspring genomes are generated by crossover, and when new nodes or edges are created during mutation. This work explores the difference between using Xavier, Kaiming, and uniform random weight initialization methods, as well as novel Lamarckian weight inheritance methods for initializing new weights during crossover and mutation operations.
  arXiv  Detail & Related papers  (2020-09-21T07:06:47Z)
• Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks [50.42141893913188]
  We study a distributed variable for large-scale AUC for a neural network as with a deep neural network. Our model requires a much less number of communication rounds and still a number of communication rounds in theory. Our experiments on several datasets show the effectiveness of our theory and also confirm our theory.
  arXiv  Detail & Related papers  (2020-05-05T18:08:23Z)
• Self-adaptation in non-Elitist Evolutionary Algorithms on Discrete Problems with Unknown Structure [2.28438857884398]
  Key challenge to make effective use of evolutionary algorithms is to choose appropriate settings for their parameters. Non-deterministic parameter control mechanisms adjust parameters using information obtained from the evolutionary process. Self-adaptation is a popular parameter control mechanism in evolutionary strategies.
  arXiv  Detail & Related papers  (2020-04-01T10:35:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.