Context-Value-Action Architecture for Value-Driven Large Language Model Agents

• URL: http://arxiv.org/abs/2604.05939v1
• Date: Tue, 07 Apr 2026 14:34:20 GMT
• Title: Context-Value-Action Architecture for Value-Driven Large Language Model Agents
• Abstract summary: Large Language Models (LLMs) have shown promise in simulating human behavior, yet existing agents often exhibit behavioral rigidity.<n>We propose the Context-Value-Action architecture, grounded in the Stimulus-Organism-Response (S-O-R) model and Schwartz's Theory of Basic Human Values.<n>Our approach effectively mitigates polarization while offering superior behavioral fidelity and interpretability.
• Score: 15.830984215635008
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) have shown promise in simulating human behavior, yet existing agents often exhibit behavioral rigidity, a flaw frequently masked by the self-referential bias of current "LLM-as-a-judge" evaluations. By evaluating against empirical ground truth, we reveal a counter-intuitive phenomenon: increasing the intensity of prompt-driven reasoning does not enhance fidelity but rather exacerbates value polarization, collapsing population diversity. To address this, we propose the Context-Value-Action (CVA) architecture, grounded in the Stimulus-Organism-Response (S-O-R) model and Schwartz's Theory of Basic Human Values. Unlike methods relying on self-verification, CVA decouples action generation from cognitive reasoning via a novel Value Verifier trained on authentic human data to explicitly model dynamic value activation. Experiments on CVABench, which comprises over 1.1 million real-world interaction traces, demonstrate that CVA significantly outperforms baselines. Our approach effectively mitigates polarization while offering superior behavioral fidelity and interpretability.

Related papers

• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• Bringing Value Models Back: Generative Critics for Value Modeling in LLM Reinforcement Learning [18.371323066068104]
  Credit assignment is a central challenge in reinforcement learning (RL)<n>We propose Generative Actor-Critic (GenAC) that replaces one-shot scalar value prediction with a generative critic.<n>GenAC improves value approximation, ranking reliability, and out-of-distribution generalization.
  arXiv  Detail & Related papers  (2026-04-12T15:54:11Z)
• Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces [81.41397370235102]
  We introduce OmniBehavior, the first user simulation benchmark constructed entirely from real-world data.<n>We show that current models struggle to accurately simulate complex behaviors, with performance plateauing even as context windows expand.<n>This results in the loss of individual differences and long-tail behaviors, highlighting critical directions for future high-fidelity simulation research.
  arXiv  Detail & Related papers  (2026-04-09T15:26:21Z)
• Preference learning in shades of gray: Interpretable and bias-aware reward modeling for human preferences [8.680081568962999]
  This study investigates the limits of current approaches to learning human preferences in language models.<n>We propose a feature-augmented framework to better capture the multidimensional nature of human judgment.
  arXiv  Detail & Related papers  (2026-04-01T18:26:16Z)
• VRM: Teaching Reward Models to Understand Authentic Human Preferences [39.939650821889764]
  Variational Reward Modeling is a novel framework that explicitly models the evaluation process of human preference judgments.<n>We show that VRM significantly outperforms existing methods in capturing authentic human preferences.
  arXiv  Detail & Related papers  (2026-03-05T09:12:39Z)
• Mitigating Reward Hacking in RLHF via Bayesian Non-negative Reward Modeling [49.41422138354821]
  We propose a principled reward modeling framework that integrates non-negative factor analysis into the Bradley-Terry preference model.<n>BNRM represents rewards through a sparse, non-negative latent factor generative process.<n>We show that BNRM substantially mitigates reward over-optimization, improves robustness under distribution shifts, and yields more interpretable reward decompositions than strong baselines.
  arXiv  Detail & Related papers  (2026-02-11T08:14:11Z)
• Stable and Explainable Personality Trait Evaluation in Large Language Models with Internal Activations [8.157769334164232]
  Internal-activation Interpolation (PVNI) for stable and explainable personality trait evaluation in Large Language Models.<n>PVNI extracts a persona vector from the model's internal activations using contrastive direction.<n>It estimates the corresponding neutral score by interpolating along the vector as an anchor axis.
  arXiv  Detail & Related papers  (2026-01-14T19:48:59Z)
• DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarios [57.327907850766785]
  characterization of deception across realistic real-world scenarios remains underexplored.<n>We establish DeceptionBench, the first benchmark that systematically evaluates how deceptive tendencies manifest across different domains.<n>On the intrinsic dimension, we explore whether models exhibit self-interested egoistic tendencies or sycophantic behaviors that prioritize user appeasement.<n>We incorporate sustained multi-turn interaction loops to construct a more realistic simulation of real-world feedback dynamics.
  arXiv  Detail & Related papers  (2025-10-17T10:14:26Z)
• On the Fairness, Diversity and Reliability of Text-to-Image Generative Models [68.62012304574012]
  multimodal generative models have sparked critical discussions on their reliability, fairness and potential for misuse.<n>We propose an evaluation framework to assess model reliability by analyzing responses to global and local perturbations in the embedding space.<n>Our method lays the groundwork for detecting unreliable, bias-injected models and tracing the provenance of embedded biases.
  arXiv  Detail & Related papers  (2024-11-21T09:46:55Z)
• It HAS to be Subjective: Human Annotator Simulation via Zero-shot Density Estimation [15.8765167340819]
  Human annotator simulation (HAS) serves as a cost-effective substitute for human evaluation such as data annotation and system assessment. Human perception and behaviour during human evaluation exhibit inherent variability due to diverse cognitive processes and subjective interpretations. This paper introduces a novel meta-learning framework that treats HAS as a zero-shot density estimation problem.
  arXiv  Detail & Related papers  (2023-09-30T20:54:59Z)
• Heterogeneous Value Alignment Evaluation for Large Language Models [91.96728871418]
  Large Language Models (LLMs) have made it crucial to align their values with those of humans. We propose a Heterogeneous Value Alignment Evaluation (HVAE) system to assess the success of aligning LLMs with heterogeneous values.
  arXiv  Detail & Related papers  (2023-05-26T02:34:20Z)
• Less is More: Mitigate Spurious Correlations for Open-Domain Dialogue Response Generation Models by Causal Discovery [52.95935278819512]
  We conduct the first study on spurious correlations for open-domain response generation models based on a corpus CGDIALOG curated in our work. Inspired by causal discovery algorithms, we propose a novel model-agnostic method for training and inference of response generation model.
  arXiv  Detail & Related papers  (2023-03-02T06:33:48Z)
• Stateful Offline Contextual Policy Evaluation and Learning [88.9134799076718]
  We study off-policy evaluation and learning from sequential data. We formalize the relevant causal structure of problems such as dynamic personalized pricing. We show improved out-of-sample policy performance in this class of relevant problems.
  arXiv  Detail & Related papers  (2021-10-19T16:15:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.