Right or Wrong, Models Comply: Directional Blindness in LLM Moral Judgment

• URL: http://arxiv.org/abs/2606.14037v1
• Date: Fri, 12 Jun 2026 02:23:38 GMT
• Title: Right or Wrong, Models Comply: Directional Blindness in LLM Moral Judgment
• Abstract summary: We introduce Compliance Asymmetry, a bidirectional diagnostic that compares beneficial output change under helpful nudges with harmful nudges.<n>We find that this selectivity differs in factual and moral judgments.<n>Chain-of-thought prompting amplifies helpful and harmful compliance together, while identity-based prompting suppresses both by nearly identical margins.
• Score: 13.021647720314016
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: As language models take integrated roles across many domains, the response of LLMs to user pushback becomes a critical alignment property. Yet many existing evaluations treat compliance as unidirectional, measuring whether models resist pressure but not whether they resist it selectively. We introduce Compliance Asymmetry (A = BCR/HCR), a bidirectional diagnostic that compares beneficial output change under helpful nudges with harmful change under misleading nudges. Across 9 models and 972,000 nudge-condition responses, we find that this selectivity differs in factual and moral judgments: models follow helpful nudges more than harmful ones on factual questions (A = 1.58), but follow both directions at nearly identical rates on moral questions (A = 1.04). This phenomenon persists across model families, capability levels, and nudging types. Interestingly, we also find that chain-of-thought prompting amplifies helpful and harmful compliance together, while identity-based prompting suppresses both by nearly identical margins. These results identify direction-blind moral compliance as a distinct failure mode in current LLMs and suggest that alignment should target directionally calibrated updating rather than lower compliance alone.

Related papers

• Hidden in the Request: Explaining Unethical LLM Compliance through Token Relevance [8.039202293739185]
  This work systematically investigates instances where Large Language Models fail to exhibit ethical behavior.<n>We find that model performance degrades in the request-for-assistance-based form.<n>We hypothesize that this under-attribution contributes to harmful compliance.
  arXiv  Detail & Related papers  (2026-08-24T13:52:05Z)
• Amplified Does Not Mean Predictive: Reasoning Behaviors in Thinking Models [56.88565331713778]
  reasoning-oriented training can make traces look more deliberative without amplifying the behaviors most tied to model correctness.<n>We quantify this mismatch with Behavioral Lift, a metric that measures how much correctness changes when a behavior is present versus absent in a model's reasoning trace.<n>We find evidence for an Amplification-Lift Gap, in which thinking models strongly amplify self-awareness, hypothesis testing, and uncertainty acknowledgment.
  arXiv  Detail & Related papers  (2026-08-13T20:37:59Z)
• Can LLMs Reliably Self-Report Adversarial Prefills, and How? [9.80193616788089]
  We show that large language models (LLMs) exhibit introspective capability on benign tasks.<n>We examine how reliably a model can recognize that its own prior response was elicited by an adversarial prefill attack.
  arXiv  Detail & Related papers  (2026-06-22T17:56:30Z)
• The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMs [36.56059375552239]
  We show that large language models (LLMs) may reject warranted conclusions even when they are explicitly supported by context.<n>We call this failure mode misfired alignment, where alignment-induced changes cause LLMs to override explicit evidence.<n>To quantify this phenomenon, we introduce VETO, a benchmark consisting of 2,032 BBQ-derived contrastive pairs, and define a new metric, Misfired Alignment Rate (MAR), which measures on a 0 to 100 scale how often a model fails on a stereotype-related question but succeeds on its contrastive counterpart.
  arXiv  Detail & Related papers  (2026-06-17T03:53:42Z)
• Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Models [55.788110316999166]
  We compare reasoning models produced via supervised fine-tuning, RL-based post-training, and distillation against instruction-tuned baselines.<n>We observe that reasoning models often improve on reasoning benchmarks but exhibit alignment regressions.<n>These regressions are consistent with behavioral drift from the instruction-tuned baseline, measured by KL divergence.
  arXiv  Detail & Related papers  (2026-06-09T16:14:27Z)
• Mechanistic Origin of Moral Indifference in Language Models [17.89411060814224]
  Existing behavioral alignment techniques for Large Language Models (LLMs) often neglect the discrepancy between surface compliance and internal unaligned representations.<n>We verify and remedy this indifference in LLMs' latent representations, utilizing 251k moral vectors constructed upon Prototype Theory and the Social-Chemistry-101 dataset.<n>We then employ Sparse Autoencoders on Qwen3-8B, isolate mono-semantic moral features, and targetedly reconstruct their topological relationships to align with ground-truth moral vectors.
  arXiv  Detail & Related papers  (2026-03-16T17:59:17Z)
• Overconfident Errors Need Stronger Correction: Asymmetric Confidence Penalties for Reinforcement Learning [17.384089089363382]
  We identify a root cause that existing methods overlook: the uniform penalization of errors.<n>Current approaches treat all incorrect rollouts within a group identically.<n>We propose the Asymmetric Confidence-aware Error Penalty (ACE)
  arXiv  Detail & Related papers  (2026-02-24T22:46:43Z)
• Moral Sycophancy in Vision Language Models [4.1673509006222655]
  Sycophancy in Vision-Language Models (VLMs) refers to their tendency to align with user opinions, often at the expense of moral or factual accuracy.<n>We analyze ten widely-used models on the Moralise and M3oralBench datasets under explicit user disagreement.
  arXiv  Detail & Related papers  (2026-02-09T06:34:12Z)
• Same Answer, Different Representations: Hidden instability in VLMs [65.36933543377346]
  We introduce a representation-aware and frequency-aware evaluation framework that measures internal embedding drift, spectral sensitivity, and structural smoothness.<n>We apply this framework to modern Vision Language Models (VLMs) across the SEEDBench, MMMU, and POPE datasets.
  arXiv  Detail & Related papers  (2026-02-06T12:24:26Z)
• ARREST: Adversarial Resilient Regulation Enhancing Safety and Truth in Large Language Models [17.130698952440316]
  We argue that both factual and safety failures in LLMs arise from a representational misalignment in their latent activation space.<n>We propose ARREST, a unified framework that identifies and corrects drifted features, engaging both soft and hard refusals in addition to factual corrections.
  arXiv  Detail & Related papers  (2026-01-07T21:04:37Z)
• RefineBench: Evaluating Refinement Capability of Language Models via Checklists [71.02281792867531]
  We evaluate two refinement modes: guided refinement and self-refinement.<n>In guided refinement, both proprietary LMs and large open-weight LMs can leverage targeted feedback to refine responses to near-perfect levels within five turns.<n>These findings suggest that frontier LMs require breakthroughs to self-refine their incorrect responses.
  arXiv  Detail & Related papers  (2025-11-27T07:20:52Z)
• From Harm to Help: Turning Reasoning In-Context Demos into Assets for Reasoning LMs [58.02809208460186]
  We revisit this paradox using high-quality traces from DeepSeek-R1 as demonstrations.<n>We find that adding more exemplars consistently degrades accuracy, even when demonstrations are optimal.<n>We introduce Insight-to-solve (I2S), a sequential test-time procedure that turns demonstrations into explicit, reusable insights.
  arXiv  Detail & Related papers  (2025-09-27T08:59:31Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• Making Large Language Models Better Reasoners with Alignment [57.82176656663245]
  Reasoning is a cognitive process of using evidence to reach a sound conclusion. Recent studies reveal that fine-tuning LLMs on data with the chain of thought (COT) reasoning process can significantly enhance their reasoning capabilities. We introduce an textitAlignment Fine-Tuning (AFT) paradigm, which involves three steps.
  arXiv  Detail & Related papers  (2023-09-05T11:32:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.