Controllable Value Alignment in Large Language Models through Neuron-Level Editing

• URL: http://arxiv.org/abs/2602.07356v1
• Date: Sat, 07 Feb 2026 04:35:16 GMT
• Title: Controllable Value Alignment in Large Language Models through Neuron-Level Editing
• Authors: Yonghui Yang, Junwei Li, Jilong Liu, Yicheng He, Fengbin Zhu, Weibiao Huang, Le Wu, Richang Hong, Tat-Seng Chua,
• Abstract summary: We propose NeVA, a neuron-level editing framework for controllable value alignment in large language models.<n>NeVA achieves stronger target value alignment while incurring smaller performance degradation on general capability.<n>NeVA significantly reduces the average leakage, with residual effects largely confined to semantically related value classes.
• Score: 87.83756695719667
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Aligning large language models (LLMs) with human values has become increasingly important as their influence on human behavior and decision-making expands. However, existing steering-based alignment methods suffer from limited controllability: steering a target value often unintentionally activates other, non-target values. To characterize this limitation, we introduce value leakage, a diagnostic notion that captures the unintended activation of non-target values during value steering, along with a normalized leakage metric grounded in Schwartz's value theory. In light of this analysis, we propose NeVA, a neuron-level editing framework for controllable value alignment in LLMs. NeVA identifies sparse, value-relevant neurons and performs inference-time activation editing, enabling fine-grained control without parameter updates or retraining. Experiments show that NeVA achieves stronger target value alignment while incurring smaller performance degradation on general capability. Moreover, NeVA significantly reduces the average leakage, with residual effects largely confined to semantically related value classes. Overall, NeVA offers a more controllable and interpretable mechanism for value alignment.

Related papers

• VISA: Value Injection via Shielded Adaptation for Personalized LLM Alignment [24.492954219955788]
  We propose a closed-loop framework designed to navigate the trade-off between fine-tuning and Aligning Large Language Models (LLMs)<n> VISA features a high-precision value detector, a semantic-to-value translator, and a core value-rewriter.<n>Our experiments demonstrate that this approach enables precise control over a model's value expression while maintaining its factual consistency and general capabilities.
  arXiv  Detail & Related papers  (2026-03-05T05:12:26Z)
• Value Alignment Tax: Measuring Value Trade-offs in LLM Alignment [16.1422306417719]
  We introduce the Value Alignment Tax (VAT), a framework that measures how alignment-induced changes propagate across values.<n>VAT captures the dynamics of value expression under alignment pressure.<n>Our results show that alignment often produces uneven, structured co-movement among values.
  arXiv  Detail & Related papers  (2026-02-12T16:21:22Z)
• V-Attack: Targeting Disentangled Value Features for Controllable Adversarial Attacks on LVLMs [66.81402538540458]
  We propose V-Attack, a novel method for precise local semantic attacks.<n>V-Attack improves the attack success rate by an average of 36% over state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-11-25T11:51:17Z)
• Every Question Has Its Own Value: Reinforcement Learning with Explicit Human Values [53.72318444646282]
  We propose Reinforcement Learning with Explicit Human Values (RLEV)<n>RLEV aligns Large Language Model (LLM) optimization directly with quantifiable human value signals.<n>We show RLEV consistently outperforms correctness-only baselines across multiple RL algorithms and model scales.
  arXiv  Detail & Related papers  (2025-10-23T04:15:22Z)
• Internal Value Alignment in Large Language Models through Controlled Value Vector Activation [70.41805604556058]
  We introduce a Controlled Value Vector Activation (ConVA) method to align Large Language Models (LLMs) with human values.<n>To consistently control values without sacrificing model performance, we introduce a gated value vector activation method.<n>Experiments show that our method achieves the highest control success rate across 10 basic values without hurting LLM performance and fluency.
  arXiv  Detail & Related papers  (2025-07-15T13:48:35Z)
• Understanding How Value Neurons Shape the Generation of Specified Values in LLMs [31.185636385067152]
  Integration of large language models into societal applications has intensified concerns about their alignment with universal ethical principles.<n>Current approaches struggle to systematically interpret how values are encoded in neural architectures.<n>We introduce Value, a mechanistic interpretability framework grounded in the Schwartz Survey.
  arXiv  Detail & Related papers  (2025-05-23T10:30:09Z)
• Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective [33.19778298286475]
  We argue that a latent causal value graph underlies the value dimensions of large language models (LLMs) and that, despite alignment training, this structure remains significantly different from human value systems.<n>We leverage these causal value graphs to guide two lightweight value-steering methods: role-based prompting and sparse autoencoder (SAE) steering.<n>Experiments on Gemma-2B-IT and Llama3-8B-IT demonstrate the effectiveness and controllability of our methods.
  arXiv  Detail & Related papers  (2024-12-31T18:12:05Z)
• Heterogeneous Value Alignment Evaluation for Large Language Models [91.96728871418]
  Large Language Models (LLMs) have made it crucial to align their values with those of humans. We propose a Heterogeneous Value Alignment Evaluation (HVAE) system to assess the success of aligning LLMs with heterogeneous values.
  arXiv  Detail & Related papers  (2023-05-26T02:34:20Z)
• Offline Contextual Bandits with Overparameterized Models [52.788628474552276]
  We ask whether the same phenomenon occurs for offline contextual bandits. We show that this discrepancy is due to the emphaction-stability of their objectives. In experiments with large neural networks, this gap between action-stable value-based objectives and unstable policy-based objectives leads to significant performance differences.
  arXiv  Detail & Related papers  (2020-06-27T13:52:07Z)
• An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction [84.49035467829819]
  We show that it is possible to better manage this trade-off by optimizing a bound on the Information Bottleneck (IB) objective. Our fully unsupervised approach jointly learns an explainer that predicts sparse binary masks over sentences, and an end-task predictor that considers only the extracted rationale.
  arXiv  Detail & Related papers  (2020-05-01T23:26:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.