Smart Enough to Go Extinct? An Evolutionary Challenge to the Value of General Intelligence and Its Ethical Implications for AGI

• URL: http://arxiv.org/abs/2608.10730v1
• Date: Tue, 11 Aug 2026 09:45:35 GMT
• Title: Smart Enough to Go Extinct? An Evolutionary Challenge to the Value of General Intelligence and Its Ethical Implications for AGI
• Abstract summary: pursuit of artificial general intelligence rests on the premise that general intelligence is extraordinarily valuable.<n>We argue that, on evolutionary timescales, general intelligence's adaptive value is far from empirically established.<n>If the long-term evolutionary value of general intelligence is uncertain or negative, this raises ethical questions about engineering AGI and, more urgently, creating artificial consciousness.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The pursuit of artificial general intelligence (AGI) rests on a seemingly self-evident premise: that general intelligence, the kind of flexible, domain-general cognitive capacity exemplified by Homo sapiens, is extraordinarily valuable. This paper subjects this premise to critical scrutiny. We first present the intuitive case for the value of general intelligence before mounting an evolutionary challenge. We argue that, on evolutionary timescales, its adaptive value is far from empirically established. Numerous taxa, from cyanobacteria to horseshoe crabs, have persisted for hundreds of millions or even billions of years without anything resembling general intelligence, while Homo sapiens has existed for roughly 300,000 years and already faces self-generated existential risks. Mass extinction events do not preferentially favour cognitively sophisticated species. We argue that general intelligence may be the only biological strategy that generates existential threats to the species possessing it, an existential risk paradox with no parallel among non-intelligent survival strategies. Unlike prevailing accounts of AI risk that trace the danger to misalignment, we locate it in structural features of general intelligence itself, implying that even well-aligned AGI would inherit this liability. If the long-term evolutionary value of general intelligence is uncertain or negative, this raises ethical questions about engineering AGI and, more urgently, creating artificial consciousness. Drawing on deontological ethics and the precautionary principle, we argue that this uncertainty imposes a duty of caution: if we create a new kind of intelligent being, we bear responsibility for ensuring the conditions under which it can flourish.

Related papers

• A Definition of AGI [208.25193480759026]
  The lack of a concrete definition for Artificial General Intelligence obscures the gap between today's specialized AI and human-level cognition.<n>This paper introduces a quantifiable framework to address this, defining AGI as matching the cognitive versatility and proficiency of a well-educated adult.
  arXiv  Detail & Related papers  (2025-10-21T01:28:35Z)
• P: A Universal Measure of Predictive Intelligence [0.0]
  There is no commonly agreed definition of the intelligence that AI systems are said to possess.<n>No-one has developed a practical measure that would enable us to compare the intelligence of humans, animals and AIs on a single ratio scale.<n>This paper sets out a new universal measure of intelligence that is based on the hypothesis that prediction is the most important component of intelligence.
  arXiv  Detail & Related papers  (2025-05-30T10:05:54Z)
• Evaluating Intelligence via Trial and Error [59.80426744891971]
  We introduce Survival Game as a framework to evaluate intelligence based on the number of failed attempts in a trial-and-error process.<n>When the expectation and variance of failure counts are both finite, it signals the ability to consistently find solutions to new challenges.<n>Our results show that while AI systems achieve the Autonomous Level in simple tasks, they are still far from it in more complex tasks.
  arXiv  Detail & Related papers  (2025-02-26T05:59:45Z)
• Imagining and building wise machines: The centrality of AI metacognition [78.76893632793497]
  We examine what is known about human wisdom and sketch a vision of its AI counterpart.<n>We argue that AI systems particularly struggle with metacognition.<n>We discuss how wise AI might be benchmarked, trained, and implemented.
  arXiv  Detail & Related papers  (2024-11-04T18:10:10Z)
• On a Functional Definition of Intelligence [0.0]
  Without an agreed-upon definition of intelligence, asking "is this system intelligent?"" is an untestable question. Most work on precisely capturing what we mean by "intelligence" has come from the fields of philosophy, psychology, and cognitive science. We present an argument for a purely functional, black-box definition of intelligence, distinct from how that intelligence is actually achieved.
  arXiv  Detail & Related papers  (2023-12-15T05:46:49Z)
• Artificial General Intelligence, Existential Risk, and Human Risk Perception [0.0]
  Artificial general intelligence (AGI) does not yet exist, but it is projected to reach human-level intelligence within roughly the next two decades. AGI poses an existential risk to humans because there is no reliable method for ensuring that AGI goals stay aligned with human goals. The perceived risk of a world catastrophe or extinction from AGI is greater than for other existential risks.
  arXiv  Detail & Related papers  (2023-11-15T04:57:16Z)
• The Generative AI Paradox: "What It Can Create, It May Not Understand" [81.89252713236746]
  Recent wave of generative AI has sparked excitement and concern over potentially superhuman levels of artificial intelligence. At the same time, models still show basic errors in understanding that would not be expected even in non-expert humans. This presents us with an apparent paradox: how do we reconcile seemingly superhuman capabilities with the persistence of errors that few humans would make?
  arXiv  Detail & Related papers  (2023-10-31T18:07:07Z)
• Natural Selection Favors AIs over Humans [18.750116414606698]
  We argue that the most successful AI agents will likely have undesirable traits. If such agents have intelligence that exceeds that of humans, this could lead to humanity losing control of its future. To counteract these risks and evolutionary forces, we consider interventions such as carefully designing AI agents' intrinsic motivations.
  arXiv  Detail & Related papers  (2023-03-28T17:59:12Z)
• When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment [96.77970239683475]
  AI systems need to be able to understand, interpret and predict human moral judgments and decisions. A central challenge for AI safety is capturing the flexibility of the human moral mind. We present a novel challenge set consisting of rule-breaking question answering.
  arXiv  Detail & Related papers  (2022-10-04T09:04:27Z)
• Inductive Biases for Deep Learning of Higher-Level Cognition [108.89281493851358]
  A fascinating hypothesis is that human and animal intelligence could be explained by a few principles. This work considers a larger list, focusing on those which concern mostly higher-level and sequential conscious processing. The objective of clarifying these particular principles is that they could potentially help us build AI systems benefiting from humans' abilities.
  arXiv  Detail & Related papers  (2020-11-30T18:29:25Z)
• Dynamic Cognition Applied to Value Learning in Artificial Intelligence [0.0]
  Several researchers in the area are trying to develop a robust, beneficial, and safe concept of artificial intelligence. It is of utmost importance that artificial intelligent agents have their values aligned with human values. A possible approach to this problem would be to use theoretical models such as SED.
  arXiv  Detail & Related papers  (2020-05-12T03:58:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.