Agreement Is Not Alignment: Divergent Moral Grounds in Human and LLM Ethical Judgments

• URL: http://arxiv.org/abs/2608.12368v1
• Date: Tue, 14 Jul 2026 09:24:14 GMT
• Title: Agreement Is Not Alignment: Divergent Moral Grounds in Human and LLM Ethical Judgments
• Abstract summary: Agreement with human judgments is a common proxy for evaluating the alignment of large language models (LLMs)<n>Yet agreement in final labels does not show that human annotators and models rely on the same moral grounds.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agreement with human judgments is a common proxy for evaluating the alignment of large language models (LLMs). Yet agreement in final labels does not show that human annotators and models rely on the same moral grounds. Two agents may reach the same judgment while appealing to different principles, contextual assumptions, or interpretations of the situation. We test this distinction using a curated 500-item ETHICS-derived benchmark spanning five domains of moral judgment, with new human annotator and LLM annotations of both final labels and supporting rationales. Across frontier and open model families, agreement with human annotator majority labels is often high. However, rationale-level analysis reveals systematic divergence in the moral grounds expressed by human annotators and models. In particular, models redistribute attention across categories such as harm, respect, promise-keeping, justice, desert, and excuse relevance, even when their final labels match the human annotator majority. Our results show that agreement should not be treated as equivalent to alignment. Label-based evaluation can therefore be misleadingly reassuring unless complemented by analysis of the reasons, principles, and moral priorities expressed in model judgments.

Related papers

• CORDA: A Benchmark for Hierarchical Harm-Centric Moral Reasoning in Large Language Models [10.221486703870996]
  Key question in moral judgement is how people decide what matters most when moral principles conflict.<n>Current evaluations of large language models (LLMs) remain limited.<n>We introduce CORDA, a benchmark for evaluating hierarchical, harm-centred moral reasoning.
  arXiv  Detail & Related papers  (2026-08-08T10:56:30Z)
• Normative Robustness as a Frontier for Non-Verifiable Reasoning in LLMs [12.025454858026187]
  We propose moral reasoning as a paradigmatic subdomain of non-verifiable reasoning.<n>We simulate 48,000 user-agent moral deliberations across four frontier LLMs.<n>Our analysis indicates that models tailor not just their final verdicts but their underlying justifications to align with a user's moral viewpoint.
  arXiv  Detail & Related papers  (2026-06-10T22:37:26Z)
• Quantifying the Statistical Effect of Rubric Modifications on Human-Autorater Agreement [50.34437999083224]
  Autoraters are increasingly used for evaluation and automated moderation content.<n>There is limited statistical analysis of how modifications in a rubric presented to both humans and autoraters affect their score agreement.
  arXiv  Detail & Related papers  (2026-05-07T13:55:11Z)
• "Pull or Not to Pull?'': Investigating Moral Biases in Leading Large Language Models Across Ethical Dilemmas [11.229443362516207]
  This study presents a comprehensive empirical evaluation of 14 leading large language models (LLMs)<n>We elicited 3,780 binary decisions and natural language justifications, enabling analysis along axes of decisional assertiveness, explanation answer consistency, public moral alignment, and sensitivity to ethically irrelevant cues.<n>We advocate for moral reasoning to become a primary axis in LLM alignment, calling for standardized benchmarks that evaluate not just what LLMs decide, but how and why.
  arXiv  Detail & Related papers  (2025-08-10T10:45:16Z)
• The Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Models [36.573147909548226]
  People increasingly rely on Large Language Models (LLMs) for moral advice, which may influence humans' decisions.<n>We find that models reproduce human judgments only under high consensus; alignment deteriorates sharply when human disagreement increases.<n>To close this gap, we introduce Dynamic Moral Profiling (DMP), a Dirichlet-based sampling method that conditions model outputs on human-derived value profiles.
  arXiv  Detail & Related papers  (2025-07-23T05:26:17Z)
• Are Language Models Consequentialist or Deontological Moral Reasoners? [75.6788742799773]
  We focus on a large-scale analysis of the moral reasoning traces provided by large language models (LLMs)<n>We introduce and test a taxonomy of moral rationales to systematically classify reasoning traces according to two main normative ethical theories: consequentialism and deontology.
  arXiv  Detail & Related papers  (2025-05-27T17:51:18Z)
• ClarityEthic: Explainable Moral Judgment Utilizing Contrastive Ethical Insights from Large Language Models [30.301864398780648]
  We introduce a novel moral judgment approach called textitEthic that leverages LLMs' reasoning ability and contrastive learning to uncover relevant social norms.<n>Our method outperforms state-of-the-art approaches in moral judgment tasks.
  arXiv  Detail & Related papers  (2024-12-17T12:22:44Z)
• Exploring and steering the moral compass of Large Language Models [55.2480439325792]
  Large Language Models (LLMs) have become central to advancing automation and decision-making across various sectors. This study proposes a comprehensive comparative analysis of the most advanced LLMs to assess their moral profiles.
  arXiv  Detail & Related papers  (2024-05-27T16:49:22Z)
• Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties [68.66719970507273]
  Value pluralism is the view that multiple correct values may be held in tension with one another. As statistical learners, AI systems fit to averages by default, washing out potentially irreducible value conflicts. We introduce ValuePrism, a large-scale dataset of 218k values, rights, and duties connected to 31k human-written situations.
  arXiv  Detail & Related papers  (2023-09-02T01:24:59Z)
• Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories? [78.3738172874685]
  Making moral judgments is an essential step toward developing ethical AI systems. Prevalent approaches are mostly implemented in a bottom-up manner, which uses a large set of annotated data to train models based on crowd-sourced opinions about morality. This work proposes a flexible top-down framework to steer (Large) Language Models (LMs) to perform moral reasoning with well-established moral theories from interdisciplinary research.
  arXiv  Detail & Related papers  (2023-08-29T15:57:32Z)
• Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes [72.64975113835018]
  Motivated by descriptive ethics, we investigate a novel, data-driven approach to machine ethics. We introduce Scruples, the first large-scale dataset with 625,000 ethical judgments over 32,000 real-life anecdotes. Our dataset presents a major challenge to state-of-the-art neural language models, leaving significant room for improvement.
  arXiv  Detail & Related papers  (2020-08-20T17:34:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.