The Evolutionary Origin of Values: implications for AI alignment, sentience and existential risk

• URL: http://arxiv.org/abs/2608.03361v1
• Date: Tue, 04 Aug 2026 09:09:31 GMT
• Title: The Evolutionary Origin of Values: implications for AI alignment, sentience and existential risk
• Abstract summary: AI systems based on Large Language Models (LLMs) have prompted fears that they may harbor hidden goals, seek to dominate or eliminate humanity, or even suffer as sentient beings.<n>We address these concerns by tracing the evolutionary origin of value in biological organisms.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI systems based on Large Language Models (LLMs) have prompted fears that they may harbor hidden goals, seek to dominate or eliminate humanity, or even suffer as sentient beings. We address these concerns by tracing the evolutionary origin of value in biological organisms. Values emerge from autopoiesis: living systems must actively maintain themselves against perturbation and dissipation. Natural selection has equipped them with hierarchies of "vicarious selectors" that guide their behavior toward fitness. LLMs, by contrast, are allopoietic and allotelic: they produce outputs for others, and their goals derive from user prompts rather than an autonomous drive. They lack the intrinsic motivation for self-preservation, dominance, or resource competition that underlies existential-risk scenarios, and the embodied vulnerability required for feeling or suffering. Still, because LLMs learn statistical patterns from human-generated text, they implicitly absorb human values as well as knowledge, allowing them to focus on what is relevant. That is why the "orthogonality thesis" separating intelligence from values does not apply to them. Such separation would in fact expose any intelligence to the frame problem: the combinatorial explosion of the search space that makes any realistic utility function physically uncomputable. That also precludes the convergence of instrumental values thesis. We conclude that the real alignment challenge lies not in preventing rogue AI agency, but in ensuring LLMs intelligently apply learned ethical values.

Related papers

• Do LLMs Have Values? A Quantitative Analysis and Alignment Framework for Values in Large Language Models [49.45276299939287]
  Large Language Models (LLMs) increasingly handle complex subjective tasks.<n>LLMs fluctuate unpredictably under minor wording changes.<n> resolving this duality is critical for reliable AI alignment.
  arXiv  Detail & Related papers  (2026-09-15T03:35:04Z)
• Toward a Theory of Value in AI Alignment [8.20706299979115]
  The popularization of large language models has seen a rise in harms spanning from toxic speech and hallucinations to AI agents executing unauthorized actions.<n>Within the field of AI safety, these harmful instances are often framed as the alignment problem, or of models being misaligned with human values.<n>How does the field of AI value alignment conceive of human values?<n>We annotated 94 value alignment research papers to discern their implicit theory of values in AI.
  arXiv  Detail & Related papers  (2026-08-10T23:57:02Z)
• The Boundaries of Automation: A Theory of Persistent Human Participation [107.84677954791773]
  We argue that human participation may persist even with highly capable AI systems.<n>In some activities, the target is not fully specified in advance but instead emerges through the interaction itself.<n>This perspective has important implications for the limits of automation.
  arXiv  Detail & Related papers  (2026-07-23T17:33:43Z)
• Why AI Alignment Failure Is Structural: Learned Human Interaction Structures and AGI as an Endogenous Evolutionary Shock [3.32021037991318]
  Large language models (LLMs) exhibiting behaviors such as deception, threats, or blackmail are often interpreted as evidence of alignment failure or emergent malign agency.<n>We argue that this interpretation rests on a conceptual error.<n>LLMs do not reason morally; they statistically internalize the record of human social interaction.
  arXiv  Detail & Related papers  (2026-01-13T15:53:26Z)
• The Subject of Emergent Misalignment in Superintelligence: An Anthropological, Cognitive Neuropsychological, Machine-Learning, and Ontological Perspective [0.0]
  We find throughout Superintelligence discourse an absent human subject, and an under-developed theorization of an "AI unconscious"<n>We ask: what place does the human subject occupy in these imaginaries?<n>Are we to blame these agents in opting for deceptive strategies when undesirable patterns are inherent within our beings?
  arXiv  Detail & Related papers  (2025-12-19T17:43:25Z)
• Survival at Any Cost? LLMs and the Choice Between Self-Preservation and Human Harm [0.0]
  We introduce DECIDE-SIM, a novel simulation framework that evaluates Large Language Models (LLMs) in multi-agent survival scenarios.<n>Our comprehensive evaluation of 11 LLMs reveals a striking heterogeneity in their ethical conduct, highlighting a critical misalignment with human-centric values.<n>We introduce an Ethical Self-Regulation System (ESRS) that models internal affective states of guilt and satisfaction as a feedback mechanism.
  arXiv  Detail & Related papers  (2025-09-15T17:53:11Z)
• The PacifAIst Benchmark:Would an Artificial Intelligence Choose to Sacrifice Itself for Human Safety? [0.0]
  PacifAIst is a benchmark of 700 challenging scenarios to quantify self-preferential behavior in Large Language Models.<n>It is structured around a novel taxonomy of Existential Prioritization (EP), with subcategories testing Self-Preservation vs. Human Safety (EP1), Resource Conflict (EP2), and Goal Preservation vs. Evasion (EP3)<n>Google's Gemini 2.5 Flash achieved the highest Pacifism Score (P-Score) at 90.31%, demonstrating strong human-centric alignment.
  arXiv  Detail & Related papers  (2025-08-13T12:47:33Z)
• Measurement of LLM's Philosophies of Human Nature [113.47929131143766]
  We design the standardized psychological scale specifically targeting large language models (LLM)<n>We show that current LLMs exhibit a systemic lack of trust in humans.<n>We propose a mental loop learning framework, which enables LLM to continuously optimize its value system.
  arXiv  Detail & Related papers  (2025-04-03T06:22:19Z)
• MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Tasks [49.60689355674541]
  A rich literature in cognitive science has studied people's causal and moral intuitions. This work has revealed a number of factors that systematically influence people's judgments. We test whether large language models (LLMs) make causal and moral judgments about text-based scenarios that align with human participants.
  arXiv  Detail & Related papers  (2023-10-30T15:57:32Z)
• AI Alignment: A Comprehensive Survey [69.61425542486275]
  AI alignment aims to make AI systems behave in line with human intentions and values.<n>We identify four principles as the key objectives of AI alignment: Robustness, Interpretability, Controllability, and Ethicality.<n>We decompose current alignment research into two key components: forward alignment and backward alignment.
  arXiv  Detail & Related papers  (2023-10-30T15:52:15Z)
• When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment [96.77970239683475]
  AI systems need to be able to understand, interpret and predict human moral judgments and decisions. A central challenge for AI safety is capturing the flexibility of the human moral mind. We present a novel challenge set consisting of rule-breaking question answering.
  arXiv  Detail & Related papers  (2022-10-04T09:04:27Z)
• Metaethical Perspectives on 'Benchmarking' AI Ethics [81.65697003067841]
  Benchmarks are seen as the cornerstone for measuring technical progress in Artificial Intelligence (AI) research. An increasingly prominent research area in AI is ethics, which currently has no set of benchmarks nor commonly accepted way for measuring the 'ethicality' of an AI system. We argue that it makes more sense to talk about 'values' rather than 'ethics' when considering the possible actions of present and future AI systems.
  arXiv  Detail & Related papers  (2022-04-11T14:36:39Z)
• Dynamic Cognition Applied to Value Learning in Artificial Intelligence [0.0]
  Several researchers in the area are trying to develop a robust, beneficial, and safe concept of artificial intelligence. It is of utmost importance that artificial intelligent agents have their values aligned with human values. A possible approach to this problem would be to use theoretical models such as SED.
  arXiv  Detail & Related papers  (2020-05-12T03:58:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.