Two Confounds in Cross-Model Value Comparison: Response Determinism and the Access Harness

• URL: http://arxiv.org/abs/2607.10202v1
• Date: Sat, 11 Jul 2026 08:26:48 GMT
• Title: Two Confounds in Cross-Model Value Comparison: Response Determinism and the Access Harness
• Abstract summary: Cross-model comparisons read divergence in value dispositions as evidence that language models hold individuated values.<n>We introduce a separation protocol -- no-rule value dilemmas with counterbalanced, repeated forced-choice measurement and a determinism index.<n>We find determinism varies substantially (0.66-0.95 among engaging models)
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cross-model comparisons read divergence in value dispositions as evidence that language models hold individuated values. Under single-draw measurement this conflates two quantities: a difference in central tendency (a genuine value difference) and a difference in response determinism (how sharply a model commits to a forced choice). We introduce a separation protocol -- no-rule value dilemmas with counterbalanced, repeated forced-choice measurement and a determinism index -- and a determinism-corrected decomposition that splits an apparent cross-model distance into a direction-flip component (genuine disagreement) and a same-side-more-extreme component we label determinism. Across nine models, determinism varies substantially (0.66-0.95 among engaging models); whether it is a per-model trait or tracks provider and scale is a question our method makes measurable but our sample leaves open. Correcting for determinism shrinks apparent individuation, while a few cross-family disagreements survive a strict test. We then isolate a second confound: the access harness serving each model. Re-collecting the same models through raw provider APIs, we find the deployment client shifts a model's value profile substantially and client-specifically: one subscription CLI moves a profile by 0.31, flips four of eighteen items, and inflates the flagship's apparent softness (0.34 via CLI vs 0.66 via raw API), whereas another provider's client is clean, confounding provider family with access client. The harness is a value-shaping layer: a base model that refuses one-in-ten forced choices is made compliant by an agent system prompt, established causally in a white-box control. An audit ranking models by single-draw value distance thus ranks a determinism-inflated quantity, confounded further by the client used. We contribute the decomposition and identify the deployment harness as a distinct value confound.

Related papers

• Nous: An Attempt to Extract and Inject the Cognition Behind Prediction-Market Behavior [0.0]
  We ask whether human cognitive diversity can be recovered from behavior and transferred to LLM agents.<n>Nous extracts a structured eight-dimension behavioral profile from real Polymarket trading activity and injects it into agents through prompts.<n>We position Nous as measuring the cognitive-monoculture problem and the limits of a prompt-level remedy.
  arXiv  Detail & Related papers  (2026-06-11T08:18:25Z)
• CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs [100.38986535324284]
  We introduce textbfCIAware-Bench, a benchmark for measuring textbfcontrol textbfintervention (CI) awareness across frontier models.<n>CIAware-Bench tests whether models can distinguish their own trajectories from those modified by a control intervention.
  arXiv  Detail & Related papers  (2026-06-09T16:24:16Z)
• Superficial Beliefs in LLM Decision-Making [17.52079171010035]
  We ask whether large language models (LLMs) merely imitate rationales when choosing between two options.<n>We compare the attribute a model says mattered most with the attribute that best explains its choice under a behavioural model fit to prior decisions.<n>We interpret this as evidence for superficial belief'' in LLM decision-making.
  arXiv  Detail & Related papers  (2026-06-09T15:54:35Z)
• Traits Run Deeper: Trait-Specific Asymmetric Fusion for Personality Assessment [53.50824432699408]
  Traits Run Deeper is a novel personality assessment framework.<n>It consists of three components: the Multimodal Foundation Representation (MFR), the Trait-Specific Modality Fusion (TSMF) and the Distribution-Calibrated Personality Regression (DCPR)
  arXiv  Detail & Related papers  (2026-06-09T06:38:36Z)
• LLMs Show No Signs Of Individuated Metacognition [0.023227405857540805]
  We decompose binary confidence judgements from 20 Large Language Models.<n>We ask whether two models that differ in confidence also differ in performance.<n>We find no evidence for significant verbalised individuated metacognition in any tested domain.
  arXiv  Detail & Related papers  (2026-05-22T23:54:33Z)
• Cross-Model Disagreement as a Label-Free Correctness Signal [10.66607150500579]
  Cross-model disagreement is a correctness indicator that can be dropped into existing production systems.<n>No generation from the verifying model is required, and no correctness labels are needed.<n>Results establish cross-model disagreement as a practical, training-free approach to label-free correctness estimation.
  arXiv  Detail & Related papers  (2026-03-26T13:46:22Z)
• Dependence-Aware Label Aggregation for LLM-as-a-Judge via Ising Models [55.94503936470247]
  Large-scale AI evaluation increasingly relies on aggregating binary judgments from $K$ annotators, including judges.<n>Most classical methods assume annotators are conditionally independent given the true label $Yin0,1$, an assumption often violated by LLM judges.<n>We study label aggregation through a hierarchy of dependence-aware models based on Ising graphical models and latent factors.
  arXiv  Detail & Related papers  (2026-01-29T21:26:50Z)
• Chain-of-Thought Unfaithfulness as Disguised Accuracy [0.0]
  Chain-of-Thought (CoT) generations align with a large language model's (LLM) internal computations. A metric that measures a model's dependence on its CoT for producing an answer is proposed.
  arXiv  Detail & Related papers  (2024-02-22T17:23:53Z)
• Confidence-Based Model Selection: When to Take Shortcuts for Subpopulation Shifts [119.22672589020394]
  We propose COnfidence-baSed MOdel Selection (CosMoS), where model confidence can effectively guide model selection. We evaluate CosMoS on four datasets with spurious correlations, each with multiple test sets with varying levels of data distribution shift.
  arXiv  Detail & Related papers  (2023-06-19T18:48:15Z)
• Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization [73.04187954213471]
  We introduce a unified learning approach to simultaneously modeling the coarse- and fine-grained retrieval. The proposed method has achieved +4.03%, +3.38%, and +2.40% Recall@50 accuracy over a strong baseline.
  arXiv  Detail & Related papers  (2022-11-14T14:25:40Z)
• AvgOut: A Simple Output-Probability Measure to Eliminate Dull Responses [97.50616524350123]
  We build dialogue models that are dynamically aware of what utterances or tokens are dull without any feature-engineering. The first model, MinAvgOut, directly maximizes the diversity score through the output distributions of each batch. The second model, Label Fine-Tuning (LFT), prepends to the source sequence a label continuously scaled by the diversity score to control the diversity level. The third model, RL, adopts Reinforcement Learning and treats the diversity score as a reward signal.
  arXiv  Detail & Related papers  (2020-01-15T18:32:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.