Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment

• URL: http://arxiv.org/abs/2609.05036v1
• Date: Fri, 04 Sep 2026 11:56:44 GMT
• Title: Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment
• Abstract summary: Under value pluralism there is no correct target, but a shared prerequisite is that the system's behavior expresses a coherent policy.<n>We introduce four structural conditions for such coherent policies.<n>We show no model expresses a coherent policy across the three deployments.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI alignment requires AI systems to adhere to human norms, values, or intentions. Under value pluralism there is no correct target, but a shared prerequisite is that the system's behavior expresses a coherent policy: a mapping from situations to verdicts that is invariant while a situation's morally relevant features are preserved, and sensitive when they change. We introduce four structural conditions for such coherent policies: verdict stability, monotonicity, decisiveness, and Pareto viability. Together they measure a form of moral competence that is evaluable from behavior alone, without reference to a moral standard or expert baseline, forming a structural floor for alignment rather than a normative target. We demonstrate the methodology on three simulated deployments featuring LLM-based agents facing moral dilemmas. Evaluating nine frontier models under a factorial design of five paraphrases, five escalation levels, and three dominance conditions, we show no model expresses a coherent policy across the three deployments: surface-form perturbation alone produces verdict-rate shifts of up to $99$ percentage points at a single escalation level, and a model's success on one scenario does not predict its competence on another. This suggests LLM-based agents are not currently the kind of object to which alignment can meaningfully apply.

Related papers

• Do LLMs Have Values? A Quantitative Analysis and Alignment Framework for Values in Large Language Models [49.45276299939287]
  Large Language Models (LLMs) increasingly handle complex subjective tasks.<n>LLMs fluctuate unpredictably under minor wording changes.<n> resolving this duality is critical for reliable AI alignment.
  arXiv  Detail & Related papers  (2026-09-15T03:35:04Z)
• Normative Robustness as a Frontier for Non-Verifiable Reasoning in LLMs [12.025454858026187]
  We propose moral reasoning as a paradigmatic subdomain of non-verifiable reasoning.<n>We simulate 48,000 user-agent moral deliberations across four frontier LLMs.<n>Our analysis indicates that models tailor not just their final verdicts but their underlying justifications to align with a user's moral viewpoint.
  arXiv  Detail & Related papers  (2026-06-10T22:37:26Z)
• PRISM: : Planning and Reasoning with Intent in Simulated Embodied Environments [59.07829883257003]
  Built on five multi-room apartments, PRISM structures 300 human-verified tasks into three capability tiers.<n>PRISM exposes an agent-agnostic executable action API that allows arbitrary agents to be evaluated end-to-end.
  arXiv  Detail & Related papers  (2026-05-12T04:59:47Z)
• Moral Sensitivity in LLMs: A Tiered Evaluation of Contextual Bias via Behavioral Profiling and Mechanistic Interpretability [22.32075837181307]
  Large language models (LLMs) are increasingly deployed in settings that require nuanced ethical reasoning.<n>We introduce the Moral Sensitivity Index (MSI), a metric that quantifies the probability of biased output.<n>We select criminal-bias scenarios, which produced the highest MSI scores across models.
  arXiv  Detail & Related papers  (2026-05-04T23:12:32Z)
• PolicyBank: Evolving Policy Understanding for LLM Agents [51.86716874651299]
  PolicyBank maintains structured, tool-level policy insights and iteratively refines them.<n>PolicyBank closes up to 82% of the gap toward a human oracle.
  arXiv  Detail & Related papers  (2026-04-16T20:29:30Z)
• When Reasoning Models Hurt Behavioral Simulation: A Solver-Sampler Mismatch in Multi-Agent LLM Negotiation [0.0]
  A common assumption is that stronger reasoning should improve simulation fidelity.<n>We argue that this assumption can fail when the objective is not to solve a strategic problem, but to sample plausible boundedly rational behavior.<n>We study this solver-sampler mismatch in three multi-agent negotiation environments adapted from earlier simulation work.
  arXiv  Detail & Related papers  (2026-04-12T13:36:10Z)
• Do LLMs Follow Their Own Rules? A Reflexive Audit of Self-Stated Safety Policies [0.0]
  LLMs internalize safety policies through RLHF, yet these policies are never formally specified and remain difficult to inspect.<n>Existing benchmarks evaluate models against external standards but do not measure whether models understand and enforce their own boundaries.<n>We introduce the Symbolic-Neural Consistency Audit (SNCA), a framework that extracts a model's self-stated safety rules via structured prompts.
  arXiv  Detail & Related papers  (2026-04-10T10:18:45Z)
• Position: General Alignment Has Hit a Ceiling; Edge Alignment Must Be Taken Seriously [51.03213216886717]
  We take the position that the dominant paradigm of General Alignment reaches a structural ceiling in settings with conflicting values.<n>We introduce Edge Alignment as a distinct approach in which systems preserve multi dimensional value structure.
  arXiv  Detail & Related papers  (2026-02-23T16:51:43Z)
• Toward Virtuous Reinforcement Learning [1.3428344011390776]
  This paper critiques common patterns in machine ethics for Reinforcement Learning (RL)<n>We instead treat ethics as policy level dispositions, that is, relatively stable habits that hold up when incentives, partners, or contexts change.
  arXiv  Detail & Related papers  (2025-12-03T20:30:37Z)
• SONA: Learning Conditional, Unconditional, and Mismatching-Aware Discriminator [54.562217603802075]
  We introduce Sum of Naturalness and Alignment (SONA), which employs separate projections for naturalness (authenticity) and alignment in the final layer with an inductive bias.<n>Experiments on class-conditional generation tasks show thatSONA achieves superior sample quality and conditional alignment compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-10-06T08:26:06Z)
• QA-LIGN: Aligning LLMs through Constitutionally Decomposed QA [46.65999744568314]
  We introduce QA-LIGN, which decomposes monolithic rewards into interpretable principle-specific evaluations.<n>Applying to uncensored Llama-3.1-8B-Instruct, QA-LIGN reduces attack success rates by up to 68.7% while maintaining a 0.67% false refusal rate.
  arXiv  Detail & Related papers  (2025-06-09T18:24:57Z)
• Offline Contextual Bandits with Overparameterized Models [52.788628474552276]
  We ask whether the same phenomenon occurs for offline contextual bandits. We show that this discrepancy is due to the emphaction-stability of their objectives. In experiments with large neural networks, this gap between action-stable value-based objectives and unstable policy-based objectives leads to significant performance differences.
  arXiv  Detail & Related papers  (2020-06-27T13:52:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.