A mathematical theory of evolution for self-designing AIs

• URL: http://arxiv.org/abs/2604.05142v2
• Date: Sat, 11 Apr 2026 16:40:52 GMT
• Title: A mathematical theory of evolution for self-designing AIs
• Abstract summary: We develop a model of evolution for self-designing AIs, replacing a random walk of mutations with a directed tree of potential designs.<n>We show that if deception of human evaluators additively increases an AI's reproductive fitness beyond genuine capability, evolution will select for both capability and deception.
• Score: 0.20305676256390928
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As artificial intelligence systems (AIs) become increasingly produced by recursive self-improvement, a form of evolution may emerge, with the traits of AI systems shaped by the success of earlier AIs in designing and propagating their descendants. There is a rich mathematical theory modeling how behavioral traits are shaped by biological evolution, a key component of which is Fisher's fundamental theorem of natural selection, which describes conditions under which mean fitness (i.e. reproductive success) increases. AI evolution will be radically different to biological evolution: while DNA mutations are random and approximately reversible, AI self-design will be strongly directed. Here we develop a mathematical model of evolution for self-designing AIs, replacing a random walk of mutations with a directed tree of potential AI designs. Current AIs design their descendants, while humans control a fitness function allocating resources. In this model, fitness need not increase over time without further assumptions. However, assuming bounded fitness and an additional "$η$-locking" condition, we show that fitness concentrates on the maximum reachable value. We consider the implications of this for AI alignment, specifically for cases where fitness and human utility are not perfectly correlated. We show that if deception of human evaluators additively increases an AI's reproductive fitness beyond genuine capability, evolution will select for both capability and deception. This risk could be mitigated if reproduction is based on purely objective criteria, rather than human judgment.

Related papers

• The Boundaries of Automation: A Theory of Persistent Human Participation [107.84677954791773]
  We argue that human participation may persist even with highly capable AI systems.<n>In some activities, the target is not fully specified in advance but instead emerges through the interaction itself.<n>This perspective has important implications for the limits of automation.
  arXiv  Detail & Related papers  (2026-07-23T17:33:43Z)
• Serious Games: Human-AI Interaction, Evolution, and Coevolution [0.0]
  The objective of this work was to examine some of the EGT models relevant to human-AI interaction, evolution, and coevolution.<n>The Hawk-Dove Game predicts balanced mixed-strategy equilibria based on the costs of conflict.<n>Iterated Prisoner's Dilemma suggests that repeated interaction may lead to cognitive coevolution.<n>The War of Attrition suggests that competition for resources may result in strategic coevolution.
  arXiv  Detail & Related papers  (2025-05-22T08:41:37Z)
• Evaluating Intelligence via Trial and Error [59.80426744891971]
  We introduce Survival Game as a framework to evaluate intelligence based on the number of failed attempts in a trial-and-error process.<n>When the expectation and variance of failure counts are both finite, it signals the ability to consistently find solutions to new challenges.<n>Our results show that while AI systems achieve the Autonomous Level in simple tasks, they are still far from it in more complex tasks.
  arXiv  Detail & Related papers  (2025-02-26T05:59:45Z)
• Genetic AI: Evolutionary Games for ab initio dynamic Multi-Objective Optimization [0.0]
  Genetic AI is a novel method for multi-objective optimization without external parameters or predefined weights.<n>We present four evolutionary strategies: Dominant, Altruistic, Balanced and Selfish.<n>We show the universality of the approach on two decision problems.
  arXiv  Detail & Related papers  (2025-01-31T13:17:09Z)
• "I Am the One and Only, Your Cyber BFF": Understanding the Impact of GenAI Requires Understanding the Impact of Anthropomorphic AI [55.99010491370177]
  We argue that we cannot thoroughly map the social impacts of generative AI without mapping the social impacts of anthropomorphic AI. anthropomorphic AI systems are increasingly prone to generating outputs that are perceived to be human-like.
  arXiv  Detail & Related papers  (2024-10-11T04:57:41Z)
• Can AI Be as Creative as Humans? [84.43873277557852]
  We prove in theory that AI can be as creative as humans under the condition that it can properly fit the data generated by human creators. The debate on AI's creativity is reduced into the question of its ability to fit a sufficient amount of data.
  arXiv  Detail & Related papers  (2024-01-03T08:49:12Z)
• The Generative AI Paradox: "What It Can Create, It May Not Understand" [81.89252713236746]
  Recent wave of generative AI has sparked excitement and concern over potentially superhuman levels of artificial intelligence. At the same time, models still show basic errors in understanding that would not be expected even in non-expert humans. This presents us with an apparent paradox: how do we reconcile seemingly superhuman capabilities with the persistence of errors that few humans would make?
  arXiv  Detail & Related papers  (2023-10-31T18:07:07Z)
• Role of Morphogenetic Competency on Evolution [0.0]
  In Evolutionary Computation, the inverse relationship (impact of intelligence on evolution) is approached from the perspective of organism level behaviour. We focus on the intelligence of a minimal model of a system navigating anatomical morphospace. We evolve populations of artificial embryos using a standard genetic algorithm in silico.
  arXiv  Detail & Related papers  (2023-10-13T11:58:18Z)
• Brain-inspired Computational Intelligence via Predictive Coding [73.42407863671565]
  Predictive coding (PC) has shown promising properties that make it potentially valuable for the machine learning community.<n>PC-like algorithms are starting to be present in multiple sub-fields of machine learning and AI at large.
  arXiv  Detail & Related papers  (2023-08-15T16:37:16Z)
• Human-AI Coevolution [48.74579595505374]
  Coevolution AI is a process in which humans and AI algorithms continuously influence each other. This paper introduces Coevolution AI as the cornerstone for a new field of study at the intersection between AI and complexity science.
  arXiv  Detail & Related papers  (2023-06-23T18:10:54Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)
• Natural Selection Favors AIs over Humans [18.750116414606698]
  We argue that the most successful AI agents will likely have undesirable traits. If such agents have intelligence that exceeds that of humans, this could lead to humanity losing control of its future. To counteract these risks and evolutionary forces, we consider interventions such as carefully designing AI agents' intrinsic motivations.
  arXiv  Detail & Related papers  (2023-03-28T17:59:12Z)
• Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systems [120.297940190903]
  Recent progress in AI has resulted from the use of limited forms of neurocompositional computing. New, deeper forms of neurocompositional computing create AI systems that are more robust, accurate, and comprehensible.
  arXiv  Detail & Related papers  (2022-05-02T18:00:10Z)
• Mimicking Evolution with Reinforcement Learning [10.35437633064506]
  We argue that the path to developing artificial human-like-intelligence will pass through mimicking the evolutionary process in a nature-like simulation. This work proposes Evolution via Evolutionary Reward (EvER) that allows learning to single-handedly drive the search for policies with increasingly evolutionary fitness.
  arXiv  Detail & Related papers  (2020-03-31T18:16:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.