The Librarian Who Refused to Code: Model-Dependent Identity Enactment in LLM Code Generation

• URL: http://arxiv.org/abs/2607.17420v1
• Date: Sun, 19 Jul 2026 21:47:58 GMT
• Title: The Librarian Who Refused to Code: Model-Dependent Identity Enactment in LLM Code Generation
• Abstract summary: We tested four prompt conditions (no persona, two engineer personas, and a research-librarian persona), 12 code-generation tasks, two frontier models, and five runs per cell.<n>Under pre-registered mixed-effects analysis, the condition-by-model interaction was significant for provider-reported output tokens.<n>On Claude Opus, the minimalist engineer persona reduced visible output by 30% without improving correctness, while the thorough engineer persona increased output without a correctness gain.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Biographical personas are widely used in system prompts, but their effects on code generation are rarely evaluated under controlled, pre-registered conditions. We tested four prompt conditions (no persona, two engineer personas, and a research-librarian persona), 12 code-generation tasks, two frontier models, and five runs per cell (480 completions). Persona effects differed between the two tested models. Under the pre-registered mixed-effects analysis, the condition-by-model interaction was significant for provider-reported output tokens; a post-hoc visible-character measure showed the same qualitative pattern. Six GPT-5.5 completions were length-capped and are reported separately. On Claude Opus, the minimalist engineer persona reduced visible output by 30% (33% in provider tokens) without improving correctness, while the thorough engineer persona increased output without a correctness gain. In an exploratory post-hoc analysis, the librarian persona elicited in-character disclaimers in 55 of 60 Opus responses and 12 genuine no-code responses, lowering mean correctness from 0.92 to 0.67. GPT-5.5 produced neither behavior in its 59 non-truncated responses. These results are consistent with personas acting as Model-Dependent behavioral-policy biases rather than universal quality interventions. We release raw completions, derived scores, analysis artifacts, a pre-registration document, and an execution gate log; end-to-end test-based rescoring requires an unreleased task harness.

Related papers

• NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment [54.4265627247104]
  Large language models (LLMs) are increasingly used in peer review at major AI conferences.<n>Existing benchmarks assess novelty as a single holistic score.<n>We present NovGauge, a human-anchored benchmark for fine-grained novelty assessment diagnosis.
  arXiv  Detail & Related papers  (2026-09-10T08:34:56Z)
• A Three-Tier Persona Vector for Controllable User Simulation in Agentic Evaluation [0.9918136378822991]
  We propose a three-tier persona vector with 23 operationalized dimensions.<n>We evaluate the persona model inside a synthetic data generation pipeline across 64,698 conversations.
  arXiv  Detail & Related papers  (2026-09-08T11:28:02Z)
• Marginal Fidelity Does Not Establish User Simulation in Demographic Synthetic Survey Panels: Response Contracts, Support Collapse and Conditioning Failure [51.736723807086385]
  Population-marginal agreement is evidence about an elicitation contract and an estimand obtainable without simulated respondents, not evidence of individual simulation.<n>On nine aligned model-battery pairs, a no-persona population-prevalence query averages 6.27 MAE versus 12.39 for committed panels and wins all nine comparisons.
  arXiv  Detail & Related papers  (2026-09-07T10:22:22Z)
• Judging Is Not Enumerating: Silent Omissions in LLM-Authored Acceptable Sets [13.85834524293015]
  We measure the capability that role assumes and find it lacking under the protocol the role is usually deployed with.<n>Models judge whether a candidate belongs far better than they author the set itself.
  arXiv  Detail & Related papers  (2026-08-02T05:00:44Z)
• Toward Auditable Fraud Detection: Combining Graph Features, Model Explanations, and Agentic Case Investigation [0.0]
  Fraud detection systems must scale with rising transaction volume while remaining explainable and reviewable.<n>We study a layered pipeline that combines a gradient-boosted classifier, graph-derived structural features, an autoencoder-based anomaly signal, and an investigation agent.<n>We conclude that each component contributes only under specific conditions, and that a plausible rationale from an investigation agent is not evidence of a better decision.
  arXiv  Detail & Related papers  (2026-07-21T16:37:41Z)
• Auto Research for Materials: Auditable AI-Scientist Workflows with Held-Out Transfer [13.399139714704226]
  An AI research agent can improve the score it sees without finding a modelling change that works on new materials.<n>We show that closed-loop agents can produce decisions that survive unseen evidence and code changes that can be reused across tasks and combined.
  arXiv  Detail & Related papers  (2026-07-19T07:02:44Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• One Reflection Is Not Enough: Self-Correcting Autonomous Research via Multi-Hypothesis Failure Attribution [85.65263343588026]
  We propose SAGE, a Self-correcting, Autonomous, Grounded Experimenter.<n>Its core mechanism, Multi-Hypothesis Failure Attribution (MHFA), treats recovery as a structured causal diagnosis.<n>On a 12-topic, 5-domain benchmark, SAGE increases metrics-bearing outputs from 42% to 92% over a baseline reflection.
  arXiv  Detail & Related papers  (2026-06-30T10:54:16Z)
• SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution [6.908637308550535]
  SEVA is a structured verification agent that emits evidence alignments, step-by-step reasoning chains, calibrated confidence, and a six-category error diagnosis.<n>On ClearFacts, SEVA-3B matches GPT-4o-mini (69.0 vs. 69.8 F1) while producing substantially richer, auditable output.
  arXiv  Detail & Related papers  (2026-06-29T02:37:13Z)
• Philosophical Dispositions as Behavioral Constraints for AI-Assisted Code Review: An Empirical Study [0.0]
  We present a system that constrains AI reviewer behavior through philosophical dispositions.<n>Each disposition is defined apophatically (by what it refuses to do)<n>We evaluate this system on 50 merged pull requests across 7 repositories spanning 5 programming languages.
  arXiv  Detail & Related papers  (2026-05-21T23:57:25Z)
• Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality? [74.69723255239663]
  Grounded Personality Reasoning requires MLLMs to anchor each Big Five rating in observable evidence through a chain of rating, reasoning, and grounding.<n> MM-OCEAN is produced by a multi-agent pipeline with human verification, with timestamped behavioral observations, evidence-grounded trait analyses, and seven categories of cue-grounding MCQs.<n>The analysis uncovers a striking Prejudice Gap: across the field, 51% of correct ratings are not grounded in retrieved cues, and the Holistic-Grounding Rate spans only 0-33.5%.
  arXiv  Detail & Related papers  (2026-05-21T07:42:47Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• RefineBench: Evaluating Refinement Capability of Language Models via Checklists [71.02281792867531]
  We evaluate two refinement modes: guided refinement and self-refinement.<n>In guided refinement, both proprietary LMs and large open-weight LMs can leverage targeted feedback to refine responses to near-perfect levels within five turns.<n>These findings suggest that frontier LMs require breakthroughs to self-refine their incorrect responses.
  arXiv  Detail & Related papers  (2025-11-27T07:20:52Z)
• Two-Faced Social Agents: Context Collapse in Role-Conditioned Large Language Models [0.0]
  GPT-5 exhibited complete mathematics contextual collapse and adopted a singular identity towards optimal responses.<n> Claude Sonnet 4.5 retained limited but measurable role-specific variation on the SAT items.<n>All models exhibited distinct role-conditioned affective preference, indicating that socio-affective variation can reemerge when cognitive constraints are relaxed.
  arXiv  Detail & Related papers  (2025-11-19T16:04:49Z)
• PRover: Proof Generation for Interpretable Reasoning over Rules [81.40404921232192]
  We propose a transformer-based model that answers binary questions over rule-bases and generates the corresponding proofs. Our model learns to predict nodes and edges corresponding to proof graphs in an efficient constrained training paradigm. We conduct experiments on synthetic, hand-authored, and human-paraphrased rule-bases to show promising results for QA and proof generation.
  arXiv  Detail & Related papers  (2020-10-06T15:47:53Z)
• TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task [80.38130122127882]
  TACRED is one of the largest, most widely used crowdsourced datasets in Relation Extraction (RE) In this paper, we investigate the questions: Have we reached a performance ceiling or is there still room for improvement? We find that label errors account for 8% absolute F1 test error, and that more than 50% of the examples need to be relabeled.
  arXiv  Detail & Related papers  (2020-04-30T15:07:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.