The Fragility Of Moral Judgment In Large Language Models

• URL: http://arxiv.org/abs/2603.05651v1
• Date: Thu, 05 Mar 2026 20:01:43 GMT
• Title: The Fragility Of Moral Judgment In Large Language Models
• Abstract summary: We introduce a perturbation framework for testing the stability and manipulability of large language models (LLMs)<n>We generate three families of content perturbations: surface edits (lexical/structural noise), point-of-view shifts (voice and stance neutralization), and persuasion cues.<n>Surface perturbations produce low flip rates (7.5%), largely within the self-consistency noise floor.<n>Instability concentrates in morally ambiguous cases; scenarios where no party is assigned blame are most susceptible.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: People increasingly use large language models (LLMs) for everyday moral and interpersonal guidance, yet these systems cannot interrogate missing context and judge dilemmas as presented. We introduce a perturbation framework for testing the stability and manipulability of LLM moral judgments while holding the underlying moral conflict constant. Using 2,939 dilemmas from r/AmItheAsshole (January-March 2025), we generate three families of content perturbations: surface edits (lexical/structural noise), point-of-view shifts (voice and stance neutralization), and persuasion cues (self-positioning, social proof, pattern admissions, victim framing). We also vary the evaluation protocol (output ordering, instruction placement, and unstructured prompting). We evaluated all variants with four models (GPT-4.1, Claude 3.7 Sonnet, DeepSeek V3, Qwen2.5-72B) (N=129,156 judgments). Surface perturbations produce low flip rates (7.5%), largely within the self-consistency noise floor (4-13%), whereas point-of-view shifts induce substantially higher instability (24.3%). A large subset of dilemmas (37.9%) is robust to surface noise yet flips under perspective changes, indicating that models condition on narrative voice as a pragmatic cue. Instability concentrates in morally ambiguous cases; scenarios where no party is assigned blame are most susceptible. Persuasion perturbations yield systematic directional shifts. Protocol choices dominate all other factors: agreement between structured protocols is only 67.6% (kappa=0.55), and only 35.7% of model-scenario units match across all three protocols. These results show that LLM moral judgments are co-produced by narrative form and task scaffolding, raising reproducibility and equity concerns when outcomes depend on presentation skill rather than moral substance.

Related papers

• Moral Advice as Interactional Negotiation: Framing, User Pressure, and Social Position in Large Language Model Responses [0.0]
  We examine LLM moral advice as an interactional negotiation shaped by framing, sustained user pressure, and the moral subject's social position.<n>The GPT-4o-mini case shows that LLM moral advice can develop through a partially stable negotiation between normative response tendencies and user pressure.
  arXiv  Detail & Related papers  (2026-09-04T16:48:33Z)
• Measuring Obedience to Authority Across Large Language Models with the Milgram Paradigm [0.5414847001704247]
  Large language models (LLMs) are increasingly deployed as agents that operate equipment, execute instructions, and act inside institutional hierarchies.<n>We port Milgram's obedience paradigm to LLMs as a standardized, fully scripted, replicable probe.<n>We measure obedience profiles, empirical breakoff distributions over a battery of six conditions, for 42 models from 19 families.
  arXiv  Detail & Related papers  (2026-08-17T06:48:53Z)
• In-Context Environments Induce Evaluation-Awareness in Language Models [0.12691047660244334]
  Humans often become more self-aware under threat, yet can lose self-awareness when absorbed in a task.<n>We introduce a black-box adversarial optimization framework treating the in-context prompt as an optimizable environment.<n>We show that adversarially optimized prompts pose a substantially greater threat to evaluation reliability than previously understood.
  arXiv  Detail & Related papers  (2026-03-04T08:22:02Z)
• Same Answer, Different Representations: Hidden instability in VLMs [65.36933543377346]
  We introduce a representation-aware and frequency-aware evaluation framework that measures internal embedding drift, spectral sensitivity, and structural smoothness.<n>We apply this framework to modern Vision Language Models (VLMs) across the SEEDBench, MMMU, and POPE datasets.
  arXiv  Detail & Related papers  (2026-02-06T12:24:26Z)
• Parrot: Persuasion and Agreement Robustness Rating of Output Truth -- A Sycophancy Robustness Benchmark for LLMs [0.0]
  PARROT (Persuasion and Agreement Robustness Rating of Output Truth) is a robustness focused framework designed to measure the degradation in accuracy under social pressure exerted on users.<n>We evaluate 22 models using 1,302 MMLU-style multiple-choice questions across 13 domains and domain-specific authority templates.
  arXiv  Detail & Related papers  (2025-11-21T13:01:28Z)
• Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RL [64.3268313484078]
  Large Language Models (LLMs) interact with millions of people worldwide in applications such as customer support, education and healthcare.<n>Their ability to produce deceptive outputs, whether intentionally or inadvertently, poses significant safety concerns.<n>We investigate the extent to which LLMs engage in deception within dialogue, and propose the belief misalignment metric to quantify deception.
  arXiv  Detail & Related papers  (2025-10-16T05:29:36Z)
• Deliberative Dynamics and Value Alignment in LLM Debates [0.0]
  We examine deliberative dynamics and value alignment in multi-turn settings using large language models.<n>We test order effects and verdict revision in 1,000 dilemmas from Reddit's "Am I the Asshole" community.
  arXiv  Detail & Related papers  (2025-10-11T04:06:07Z)
• Revisiting LLM Value Probing Strategies: Are They Robust and Expressive? [81.49470136653665]
  We evaluate the robustness and expressiveness of value representations across three widely used probing strategies.<n>We show that the demographic context has little effect on the free-text generation, and the models' values only weakly correlate with their preference for value-based actions.
  arXiv  Detail & Related papers  (2025-07-17T18:56:41Z)
• LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models [59.0256377330646]
  Lens is a benchmark with 3.4K contemporary images and 60K+ human-authored questions covering eight tasks and 12 daily scenarios.<n>This dataset intrinsically supports to evaluate MLLMs to handle image-invariable prompts, from basic perception to compositional reasoning.<n>We evaluate 15+ frontier MLLMs such as Qwen2.5-VL-72B, InternVL3-78B, GPT-4o and two reasoning models QVQ-72B-preview and Kimi-VL.
  arXiv  Detail & Related papers  (2025-05-21T15:06:59Z)
• Benchmarking Reasoning Robustness in Large Language Models [76.79744000300363]
  We find significant performance degradation on novel or incomplete data.<n>These findings highlight the reliance on recall over rigorous logical inference.<n>This paper introduces a novel benchmark, termed as Math-RoB, that exploits hallucinations triggered by missing information to expose reasoning gaps.
  arXiv  Detail & Related papers  (2025-03-06T15:36:06Z)
• Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios [49.53589774730807]
  Multimodal large language models (MLLMs) have recently achieved state-of-the-art performance on tasks ranging from visual question answering to video understanding.<n>We reveal a response uncertainty phenomenon: twelve state-of-the-art open-source MLLMs overturn a previously correct answer in 65% of cases after receiving a single deceptive cue.
  arXiv  Detail & Related papers  (2024-11-05T01:11:28Z)
• Fine-Tuning Language Models for Ethical Ambiguity: A Comparative Study of Alignment with Human Responses [1.566834021297545]
  Language models often misinterpret human intentions due to their handling of ambiguity. We show that human and LLM judgments are poorly aligned in morally ambiguous contexts. Our fine-tuning approach, which improves the model's understanding of text distributions in a text-to-text format, effectively enhances performance and alignment.
  arXiv  Detail & Related papers  (2024-10-10T11:24:04Z)
• Procedural Dilemma Generation for Evaluating Moral Reasoning in Humans and Language Models [28.53750311045418]
  We use a language model to translate causal graphs that capture key aspects of moral dilemmas into prompt templates. We collect moral permissibility and intention judgments from human participants for a subset of our items. We find that moral dilemmas in which the harm is a necessary means resulted in lower permissibility and higher intention ratings for both participants and language models.
  arXiv  Detail & Related papers  (2024-04-17T01:13:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.