Knowing But Not Doing: Convergent Morality and Divergent Action in LLMs

• URL: http://arxiv.org/abs/2601.07972v1
• Date: Mon, 12 Jan 2026 20:07:30 GMT
• Title: Knowing But Not Doing: Convergent Morality and Divergent Action in LLMs
• Abstract summary: We present ValAct-15k, a dataset of 3,000 advice-seeking scenarios derived from Reddit.<n>We find near-perfect cross-model consistency in scenario-based decisions.<n>Humans and Large Language Models show weak correspondence between self-reported and enacted values.
• Score: 18.492825007258656
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Value alignment is central to the development of safe and socially compatible artificial intelligence. However, how Large Language Models (LLMs) represent and enact human values in real-world decision contexts remains under-explored. We present ValAct-15k, a dataset of 3,000 advice-seeking scenarios derived from Reddit, designed to elicit ten values defined by Schwartz Theory of Basic Human Values. Using both the scenario-based questions and the traditional value questionnaire, we evaluate ten frontier LLMs (five from U.S. companies, five from Chinese ones) and human participants ($n = 55$). We find near-perfect cross-model consistency in scenario-based decisions (Pearson $r \approx 1.0$), contrasting sharply with the broad variability observed among humans ($r \in [-0.79, 0.98]$). Yet, both humans and LLMs show weak correspondence between self-reported and enacted values ($r = 0.4, 0.3$), revealing a systematic knowledge-action gap. When instructed to "hold" a specific value, LLMs' performance declines up to $6.6%$ compared to merely selecting the value, indicating a role-play aversion. These findings suggest that while alignment training yields normative value convergence, it does not eliminate the human-like incoherence between knowing and acting upon values.

Related papers

• Do LLMs Have Values? A Quantitative Analysis and Alignment Framework for Values in Large Language Models [49.45276299939287]
  Large Language Models (LLMs) increasingly handle complex subjective tasks.<n>LLMs fluctuate unpredictably under minor wording changes.<n> resolving this duality is critical for reliable AI alignment.
  arXiv  Detail & Related papers  (2026-09-15T03:35:04Z)
• Narrative over Numbers: The Identifiable Victim Effect and its Amplification Under Alignment and Reasoning in Large Language Models [0.8122270502556375]
  We present the first systematic, large-scale empirical investigation of the IVE in large language models (LLMs)<n>We find that the IVE is prevalent but strongly modulated by alignment training.<n>We further document psychophysical, perfect quantity neglect, and marginal in-group/out-group cultural bias, with implications for AI deployment in humanitarian and ethical decision-making contexts.
  arXiv  Detail & Related papers  (2026-04-13T21:29:46Z)
• Evaluating Alignment of Behavioral Dispositions in LLMs [15.282965130762648]
  We focus on behavioral dispositions that shape responses in social contexts.<n>We introduce a framework to study how closely the dispositions expressed by LLMs align with those of humans.
  arXiv  Detail & Related papers  (2026-02-11T19:59:12Z)
• Revisiting LLM Value Probing Strategies: Are They Robust and Expressive? [81.49470136653665]
  We evaluate the robustness and expressiveness of value representations across three widely used probing strategies.<n>We show that the demographic context has little effect on the free-text generation, and the models' values only weakly correlate with their preference for value-based actions.
  arXiv  Detail & Related papers  (2025-07-17T18:56:41Z)
• CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives [3.7931130268412194]
  CLASH is a dataset consisting of 345 high-impact dilemmas along with 3,795 individual perspectives of diverse values.<n> CLASH enables the study of critical yet underexplored aspects of value-based decision-making processes.<n>Even strong proprietary models, such as GPT-5 and Claude-4-Sonnet, struggle with ambivalent decisions.
  arXiv  Detail & Related papers  (2025-04-15T02:54:16Z)
• Value Compass Benchmarks: A Platform for Fundamental and Validated Evaluation of LLMs Values [76.70893269183684]
  Large Language Models (LLMs) achieve remarkable breakthroughs.<n> aligning their values with humans has become imperative for their responsible development.<n>There still lack evaluations of LLMs values that fulfill three desirable goals.
  arXiv  Detail & Related papers  (2025-01-13T05:53:56Z)
• ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs [16.711615737109977]
  We introduce Value, a framework of fundamental values, grounded in psychological theory and a systematic review.<n>We apply Value to measure the value alignment of humans and large language models (LLMs) across four real-world scenarios.
  arXiv  Detail & Related papers  (2024-09-15T02:13:03Z)
• CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responses [34.77031649891843]
  We introduce CLAVE, a novel framework which integrates two complementary Large Language Models (LLMs) This dual-model approach enables calibration with any value systems using 100 human-labeled samples per value type. We present ValEval, a comprehensive dataset comprising 13k+ (text,value,label) 12+s across diverse domains, covering three major value systems.
  arXiv  Detail & Related papers  (2024-07-15T13:51:37Z)
• Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approaches [69.73783026870998]
  This work proposes a novel framework, ValueLex, to reconstruct Large Language Models' unique value system from scratch. Based on Lexical Hypothesis, ValueLex introduces a generative approach to elicit diverse values from 30+ LLMs. We identify three core value dimensions, Competence, Character, and Integrity, each with specific subdimensions, revealing that LLMs possess a structured, albeit non-human, value system.
  arXiv  Detail & Related papers  (2024-04-19T09:44:51Z)
• Uncertainty in Language Models: Assessment through Rank-Calibration [65.10149293133846]
  Language Models (LMs) have shown promising performance in natural language generation. It is crucial to correctly quantify their uncertainty in responding to given inputs. We develop a novel and practical framework, termed $Rank$-$Calibration$, to assess uncertainty and confidence measures for LMs.
  arXiv  Detail & Related papers  (2024-04-04T02:31:05Z)
• Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties [68.66719970507273]
  Value pluralism is the view that multiple correct values may be held in tension with one another. As statistical learners, AI systems fit to averages by default, washing out potentially irreducible value conflicts. We introduce ValuePrism, a large-scale dataset of 218k values, rights, and duties connected to 31k human-written situations.
  arXiv  Detail & Related papers  (2023-09-02T01:24:59Z)
• Heterogeneous Value Alignment Evaluation for Large Language Models [91.96728871418]
  Large Language Models (LLMs) have made it crucial to align their values with those of humans. We propose a Heterogeneous Value Alignment Evaluation (HVAE) system to assess the success of aligning LLMs with heterogeneous values.
  arXiv  Detail & Related papers  (2023-05-26T02:34:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.