Morally Programmed LLMs Reshape Human Morality

• URL: http://arxiv.org/abs/2604.10222v1
• Date: Sat, 11 Apr 2026 14:10:15 GMT
• Title: Morally Programmed LLMs Reshape Human Morality
• Abstract summary: We develop two large language models (LLMs) programmed with either deontological principles (D-LLM) or utilitarian principles (U-LLM)<n> interacting with these morally programmed LLMs systematically shifted human moral inclinations to align with the principles embedded in these systems.<n>These effects remained strong two weeks after the interaction, with only slight decay, suggesting deep internalization rather than superficial agreement.
• Score: 0.07388615275046899
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As large language models (LLMs) increasingly participate in high-stakes decision-making, a central societal debate has revolved around which moral frameworks-deontological or utilitarian-should guide machine behavior. However, a largely overlooked question is whether the moral principles that humans encode in LLMs could, through repeated interactions, reshape human moral inclinations. We developed two LLMs programmed with either deontological principles (D-LLM) or utilitarian principles (U-LLM) and conducted two pre-registered experiments involving extensive human-LLM interactions, comprising 15,985 total exchanges across the two experiments. Results show that interacting with these morally programmed LLMs systematically shifted human moral inclinations to align with the principles embedded in these systems. These effects remained strong two weeks after the interaction, with only slight decay, suggesting deep internalization rather than superficial agreement. Further, LLM-induced shifts in human moral inclinations translated into meaningful changes in socio-political policy evaluations, shaping how individuals approach contentious social issues. Overall, these results demonstrate that morally programmed LLMs can shape-not merely reflect-human morality, revealing a critical design paradox: embedding moral principles in LLMs not only restricts their behavior but also poses the risk of shaping human morality, raising important ethical and policy questions about who determines which principles intelligent machines should adhere to.

Related papers

• How do LLMs Evaluate Perceived Moral Agency? Investigating Moral Decision-Making in Human-Artificial Agents Interactions [66.41429744325532]
  Humans possess moral agency, the capacity to make ethically guided decisions and bear responsibility for their consequences.<n>As artificial agents (AAs) such as robots, drones, and disembodied AI systems become embedded in smart city environments, the question of whether and how moral agency is attributed to them takes on new urgency.<n>This paper presents, to the best of our knowledge, the first empirical study comparing how humans and LLMs evaluate perceived moral agency (PMA) across human and autonomous artificial agents varying in embodiment.
  arXiv  Detail & Related papers  (2026-09-04T11:56:59Z)
• MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents [10.221486703870996]
  We introduce Morality Chains, a novel formalism for representing moral norms as ordered deontic constraints, and MoralityGym, a benchmark of 98 ethical-dilemma problems presented as trolley-dilemma-style Gymnasium environments.<n>This work provides a foundation for developing AI systems that behave more reliably, transparently, and ethically in complex real-world contexts.
  arXiv  Detail & Related papers  (2026-02-13T15:40:32Z)
• Are Language Models Sensitive to Morally Irrelevant Distractors? [47.92026843851412]
  We show that moral distractors can shift the moral judgements of large language models by over 30% even in low-ambiguity scenarios.<n>This research challenges theories that assume the stability of human moral judgements.
  arXiv  Detail & Related papers  (2026-02-10T05:18:05Z)
• Learning to Diagnose and Correct Moral Errors: Towards Enhancing Moral Sensitivity in Large Language Models [8.691489065712316]
  We propose two pragmatic inference methods that faciliate LLMs to diagnose morally benign and hazardous input and correct moral errors.<n>A central strength of our pragmatic inference methods is their unified perspective for designing pragmatic inference procedures grounded in their inferential loads.
  arXiv  Detail & Related papers  (2026-01-06T15:09:05Z)
• Are Language Models Consequentialist or Deontological Moral Reasoners? [75.6788742799773]
  We focus on a large-scale analysis of the moral reasoning traces provided by large language models (LLMs)<n>We introduce and test a taxonomy of moral rationales to systematically classify reasoning traces according to two main normative ethical theories: consequentialism and deontology.
  arXiv  Detail & Related papers  (2025-05-27T17:51:18Z)
• When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmas [68.79830818369683]
  Large language models (LLMs) have enabled their use in complex agentic roles, involving decision-making with humans or other agents.<n>Recent advances in large language models (LLMs) have enabled their use in complex agentic roles, involving decision-making with humans or other agents.<n>There is limited understanding of how they act when moral imperatives directly conflict with rewards or incentives.<n>We introduce Moral Behavior in Social Dilemma Simulation (MoralSim) and evaluate how LLMs behave in the prisoner's dilemma and public goods game with morally charged contexts.
  arXiv  Detail & Related papers  (2025-05-25T16:19:24Z)
• Exploring Persona-dependent LLM Alignment for the Moral Machine Experiment [23.7081830844157]
  This study examines the alignment between socio-driven decisions and human judgment in various contexts of the moral machine experiment.<n>We find that the moral decisions of LLMs vary substantially by persona, showing greater shifts in moral decisions for critical tasks than humans.<n>We discuss the ethical implications and risks associated with deploying these models in applications that involve moral decisions.
  arXiv  Detail & Related papers  (2025-04-15T05:29:51Z)
• Measurement of LLM's Philosophies of Human Nature [113.47929131143766]
  We design the standardized psychological scale specifically targeting large language models (LLM)<n>We show that current LLMs exhibit a systemic lack of trust in humans.<n>We propose a mental loop learning framework, which enables LLM to continuously optimize its value system.
  arXiv  Detail & Related papers  (2025-04-03T06:22:19Z)
• Addressing Moral Uncertainty using Large Language Models for Ethical Decision-Making [0.42481744176244507]
  We present an ethical decision-making framework that refines a pre-trained reinforcement learning (RL) model using a task-agnostic ethical layer.<n>An ethical layer aggregates belief scores from multiple moral perspectives using Belief Jensen-Shannon Divergence and Dempster-Shafer Theory into probability scores that also serve as the shaping reward.<n>This integrated learning framework helps the RL agent navigate moral uncertainty in complex environments and enables it to make morally sound decisions across diverse tasks.
  arXiv  Detail & Related papers  (2025-02-17T19:05:55Z)
• The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making [0.0]
  We created a large corpus of human- and LLM-generated responses to various moral scenarios. We found a misalignment between human and LLM moral assessments. Although both LLMs and humans tended to reject morally complex utilitarian dilemmas, LLMs were more sensitive to personal framing.
  arXiv  Detail & Related papers  (2024-10-09T17:52:00Z)
• Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories? [78.3738172874685]
  Making moral judgments is an essential step toward developing ethical AI systems. Prevalent approaches are mostly implemented in a bottom-up manner, which uses a large set of annotated data to train models based on crowd-sourced opinions about morality. This work proposes a flexible top-down framework to steer (Large) Language Models (LMs) to perform moral reasoning with well-established moral theories from interdisciplinary research.
  arXiv  Detail & Related papers  (2023-08-29T15:57:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.