Tracing Moral Foundations in Large Language Models

• URL: http://arxiv.org/abs/2601.05437v1
• Date: Fri, 09 Jan 2026 00:09:28 GMT
• Title: Tracing Moral Foundations in Large Language Models
• Abstract summary: We study how moral foundations are encoded, organized, and expressed within two instruction-tuned large language models.<n>We find that both models represent and distinguish moral foundations in a structured, layer-dependent way that aligns with human judgments.
• Score: 33.17451937747417
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) often produce human-like moral judgments, but it is unclear whether this reflects an internal conceptual structure or superficial ``moral mimicry.'' Using Moral Foundations Theory (MFT) as an analytic framework, we study how moral foundations are encoded, organized, and expressed within two instruction-tuned LLMs: Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct. We employ a multi-level approach combining (i) layer-wise analysis of MFT concept representations and their alignment with human moral perceptions, (ii) pretrained sparse autoencoders (SAEs) over the residual stream to identify sparse features that support moral concepts, and (iii) causal steering interventions using dense MFT vectors and sparse SAE features. We find that both models represent and distinguish moral foundations in a structured, layer-dependent way that aligns with human judgments. At a finer scale, SAE features show clear semantic links to specific foundations, suggesting partially disentangled mechanisms within shared representations. Finally, steering along either dense vectors or sparse features produces predictable shifts in foundation-relevant behavior, demonstrating a causal connection between internal representations and moral outputs. Together, our results provide mechanistic evidence that moral concepts in LLMs are distributed, layered, and partly disentangled, suggesting that pluralistic moral structure can emerge as a latent pattern from the statistical regularities of language alone.

Related papers

• Interpretable and Fair Generalized Additive Neural Networks via Multi-objective Learning [52.66717986145448]
  Interpretability and fairness are two of the most emphasized dimensions in trustworthy artificial intelligence (AI)<n>This paper focuses on neural network (NN)-based generalized additive models (GAMs), a class of self-interpretable models.
  arXiv  Detail & Related papers  (2026-09-05T07:29:42Z)
• Mechanistic Origin of Moral Indifference in Language Models [17.89411060814224]
  Existing behavioral alignment techniques for Large Language Models (LLMs) often neglect the discrepancy between surface compliance and internal unaligned representations.<n>We verify and remedy this indifference in LLMs' latent representations, utilizing 251k moral vectors constructed upon Prototype Theory and the Social-Chemistry-101 dataset.<n>We then employ Sparse Autoencoders on Qwen3-8B, isolate mono-semantic moral features, and targetedly reconstruct their topological relationships to align with ground-truth moral vectors.
  arXiv  Detail & Related papers  (2026-03-16T17:59:17Z)
• Emergent Structured Representations Support Flexible In-Context Inference in Large Language Models [77.98801218316505]
  Large language models (LLMs) exhibit emergent behaviors suggestive of human-like reasoning.<n>We investigate the internal processing of LLMs during in-context concept inference.
  arXiv  Detail & Related papers  (2026-02-08T03:14:39Z)
• Concept Component Analysis: A Principled Approach for Concept Extraction in LLMs [51.378834857406325]
  Mechanistic interpretability seeks to mitigate the issues through extracts from large language models.<n>Sparse autoencoders (SAEs) have emerged as a popular approach for extracting interpretable and monosemantic concepts.<n>We show that SAEs suffer from a fundamental theoretical ambiguity: the well-defined correspondence between LLM representations and human-interpretable concepts remains unclear.
  arXiv  Detail & Related papers  (2026-01-28T09:27:05Z)
• Pragmatic Inference for Moral Reasoning Acquisition: Generalization via Distributional Semantics [9.209639324860207]
  Moral reasoning has emerged as a promising research direction for Large Language Models (LLMs)<n>This paper investigates how LLMs can achieve generalized moral reasoning despite their reliance on distributional semantics.
  arXiv  Detail & Related papers  (2025-09-28T22:40:58Z)
• MORABLES: A Benchmark for Assessing Abstract Moral Reasoning in LLMs with Fables [50.29407048003165]
  We present MORABLES, a human-verified benchmark built from fables and short stories drawn from historical literature.<n>The main task is structured as multiple-choice questions targeting moral inference, with carefully crafted distractors that challenge models to go beyond shallow, extractive question answering.<n>Our findings show that, while larger models outperform smaller ones, they remain susceptible to adversarial manipulation and often rely on superficial patterns rather than true moral reasoning.
  arXiv  Detail & Related papers  (2025-09-15T19:06:10Z)
• LLM Assertiveness can be Mechanistically Decomposed into Emotional and Logical Components [0.17188280334580197]
  Large Language Models (LLMs) often display overconfidence, presenting information with unwarranted certainty in high-stakes contexts.<n>We use open-sourced Llama 3.2 models fine-tuned on human annotated assertiveness datasets.<n>Our analysis identifies layers most sensitive to assertiveness contrasts and reveals that high-assertive representations decompose into two sub-components of emotional and logical clusters.
  arXiv  Detail & Related papers  (2025-08-24T01:43:48Z)
• Probabilistic Aggregation and Targeted Embedding Optimization for Collective Moral Reasoning in Large Language Models [14.425718737962102]
  We propose a framework that synthesizes multiple LLMs' moral judgments into a collectively formulated moral judgment.<n>Our aggregation mechanism fuses continuous moral acceptability scores (beyond binary labels) into a collective probability.<n>Experiments on a large-scale social moral dilemma dataset show our approach builds robust consensus and improves individual model fidelity.
  arXiv  Detail & Related papers  (2025-06-17T15:22:21Z)
• Are Language Models Consequentialist or Deontological Moral Reasoners? [75.6788742799773]
  We focus on a large-scale analysis of the moral reasoning traces provided by large language models (LLMs)<n>We introduce and test a taxonomy of moral rationales to systematically classify reasoning traces according to two main normative ethical theories: consequentialism and deontology.
  arXiv  Detail & Related papers  (2025-05-27T17:51:18Z)
• From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning [63.25540801694765]
  Large Language Models (LLMs) demonstrate striking linguistic abilities, yet whether they achieve this same balance remains unclear.<n>We apply the Information Bottleneck principle to quantitatively compare how LLMs and humans navigate this compression-meaning trade-off.
  arXiv  Detail & Related papers  (2025-05-21T16:29:00Z)
• Large Multi-modal Models Can Interpret Features in Large Multi-modal Models [51.485491249693155]
  We first apply a Sparse Autoencoder to disentangle the representations into human understandable features.<n>We then present an automatic interpretation framework to interpreted the open-semantic features learned in SAE by the LMMs themselves.
  arXiv  Detail & Related papers  (2024-11-22T14:41:36Z)
• The Moral Mind(s) of Large Language Models [0.0]
  We show that large language models (LLMs) exhibit a consistent structure of moral preferences guiding their decisions.<n>Using a probabilistic rationality test, we found that at least one model from each major provider exhibited behavior consistent with approximately stable moral preferences.<n>We then estimated these utility functions and found that most models cluster around neutral moral stances.
  arXiv  Detail & Related papers  (2024-11-19T15:40:16Z)
• Guiding the PLMs with Semantic Anchors as Intermediate Supervision: Towards Interpretable Semantic Parsing [57.11806632758607]
  We propose to incorporate the current pretrained language models with a hierarchical decoder network. By taking the first-principle structures as the semantic anchors, we propose two novel intermediate supervision tasks. We conduct intensive experiments on several semantic parsing benchmarks and demonstrate that our approach can consistently outperform the baselines.
  arXiv  Detail & Related papers  (2022-10-04T07:27:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.