The Alignment Target Problem: Divergent Moral Judgments of Humans, AI Systems, and Their Designers

• URL: http://arxiv.org/abs/2604.24155v1
• Date: Mon, 27 Apr 2026 08:12:34 GMT
• Title: The Alignment Target Problem: Divergent Moral Judgments of Humans, AI Systems, and Their Designers
• Abstract summary: The quest to align machine behavior with human values raises fundamental questions about the moral frameworks that should govern AI decision-making.<n>We show that people apply meaningfully different moral standards to AI systems, to humans acting in the same situation, and to the humans who design them.
• Score: 2.5782420501870287
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The quest to align machine behavior with human values raises fundamental questions about the moral frameworks that should govern AI decision-making. Much alignment research assumes that the appropriate benchmark is how humans themselves would act in a given situation. Research into agent-type value forks has challenged this assumption by showing that people do not always hold AI systems to the same moral standards as humans. Yet this challenge is subject to two further questions: whether people evaluate AI behavior differently when its human origins are made visible, and whether people hold the humans who program AI systems to different moral standards than either the humans or the machines under evaluation. An experimental study on 1,002 U.S. adults measured moral judgments in a runaway mine train scenario, varying the subject of evaluation across four conditions: a repairman, a repair robot, a repair robot programmed by company engineers, and company engineers programming a repair robot. We find no significant variation in the moral standards applied to the repairman and the robot. However, moral judgments shifted substantially when robot actions were described as the product of human design. Participants exhibited markedly more deontological reasoning when evaluating the robot programmed by engineers or the engineers programming it, suggesting that making human design visible activates heightened moral constraints. These findings provide evidence that people apply meaningfully different moral standards to AI systems, to humans acting in the same situation, and to the humans who design them. We call this divergence the alignment target problem. Whether these plural normative standards can be reconciled into a coherent framework for AI governance in high-stakes domains remains an open question.

Related papers

• How do LLMs Evaluate Perceived Moral Agency? Investigating Moral Decision-Making in Human-Artificial Agents Interactions [66.41429744325532]
  Humans possess moral agency, the capacity to make ethically guided decisions and bear responsibility for their consequences.<n>As artificial agents (AAs) such as robots, drones, and disembodied AI systems become embedded in smart city environments, the question of whether and how moral agency is attributed to them takes on new urgency.<n>This paper presents, to the best of our knowledge, the first empirical study comparing how humans and LLMs evaluate perceived moral agency (PMA) across human and autonomous artificial agents varying in embodiment.
  arXiv  Detail & Related papers  (2026-09-04T11:56:59Z)
• Whether We Care, How We Reason: The Dual Role of Anthropomorphism and Moral Foundations in Robot Abuse [10.376222111441399]
  This study examined how anthropomorphic levels and moral foundations shape reactions to robot abuse.<n>Results revealed that anthropomorphism determines whether people extend moral consideration to robots, while moral foundations shape how they reason about such consideration.
  arXiv  Detail & Related papers  (2026-01-27T17:34:31Z)
• A roadmap for AI in robotics [55.87087746398059]
  We are witnessing growing excitement in robotics at the prospect of leveraging the potential of AI to tackle some of the outstanding barriers to the full deployment of robots in our daily lives.<n>This article offers an assessment of what AI for robotics has achieved since the 1990s and proposes a short- and medium-term research roadmap listing challenges and promises.
  arXiv  Detail & Related papers  (2025-07-26T15:18:28Z)
• A theory of appropriateness with applications to generative artificial intelligence [56.23261221948216]
  We need to understand how appropriateness guides human decision making in order to properly evaluate AI decision making and improve it.<n>This paper presents a theory of appropriateness: how it functions in human society, how it may be implemented in the brain, and what it means for responsible deployment of generative AI technology.
  arXiv  Detail & Related papers  (2024-12-26T00:54:03Z)
• The AI Double Standard: Humans Judge All AIs for the Actions of One [0.0]
  As AI proliferates, perceptions may become entangled via the moral spillover of attitudes towards one AI to attitudes towards other AIs.<n>We tested how the seemingly harmful and immoral actions of an AI or human agent spill over to attitudes towards other AIs or humans in two preregistered experiments.
  arXiv  Detail & Related papers  (2024-12-08T19:26:52Z)
• Attributions toward Artificial Agents in a modified Moral Turing Test [0.6284264304179837]
  We ask people to distinguish real human moral evaluations from those made by a popular advanced AI language model: GPT-4. A representative sample of 299 U.S. adults rated the AI's moral reasoning as superior in quality to humans' along almost all dimensions. The emergence of language models capable of producing moral responses perceived as superior in quality to humans' raises concerns that people may uncritically accept potentially harmful moral guidance from AI.
  arXiv  Detail & Related papers  (2024-04-03T13:00:47Z)
• HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation [50.616995671367704]
  We present a high-dimensional, simulated robot learning benchmark, HumanoidBench, featuring a humanoid robot equipped with dexterous hands. Our findings reveal that state-of-the-art reinforcement learning algorithms struggle with most tasks, whereas a hierarchical learning approach achieves superior performance when supported by robust low-level policies.
  arXiv  Detail & Related papers  (2024-03-15T17:45:44Z)
• Can Machines Imitate Humans? Integrative Turing-like tests for Language and Vision Demonstrate a Narrowing Gap [56.611702960809644]
  We benchmark AI's ability to imitate humans in three language tasks and three vision tasks.<n>Next, we conducted 72,191 Turing-like tests with 1,916 human judges and 10 AI judges.<n>Imitation ability showed minimal correlation with conventional AI performance metrics.
  arXiv  Detail & Related papers  (2022-11-23T16:16:52Z)
• When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment [96.77970239683475]
  AI systems need to be able to understand, interpret and predict human moral judgments and decisions. A central challenge for AI safety is capturing the flexibility of the human moral mind. We present a novel challenge set consisting of rule-breaking question answering.
  arXiv  Detail & Related papers  (2022-10-04T09:04:27Z)
• Meaningful human control over AI systems: beyond talking the talk [8.351027101823705]
  We identify four properties which AI-based systems must have to be under meaningful human control. First, a system in which humans and AI algorithms interact should have an explicitly defined domain of morally loaded situations. Second, humans and AI agents within the system should have appropriate and mutually compatible representations. Third, responsibility attributed to a human should be commensurate with that human's ability and authority to control the system.
  arXiv  Detail & Related papers  (2021-11-25T11:05:37Z)
• I Need Your Advice... Human Perceptions of Robot Moral Advising Behaviors [2.0743129221959284]
  We explore how robots should communicate in moral advising scenarios. Our results suggest that, in fact, both humans and robots are judged more positively when they provide the advice that favors the common good over an individual's life. These results raise critical new questions regarding people's moral responses to robots and the design of autonomous moral agents.
  arXiv  Detail & Related papers  (2021-04-14T16:45:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.