Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models

• URL: http://arxiv.org/abs/2607.19257v1
• Date: Tue, 21 Jul 2026 16:31:35 GMT
• Title: Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models
• Abstract summary: Two experiments cross all three factors on one held, contamination-free synthetic corpus.<n>Experiment 1 measures instruction-following decay as rule count N grows from 10 to 160, crossed with four formats and system-prompt vs. user-turn placement.<n>Experiment 2 measures recall accuracy, false-premise sycophancy, and absent-fact fabrication across a 2k-to-512k-token context ladder.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Practitioners make three prompt-design decisions with almost no controlled evidence behind them: how to format instructions and context (markdown, plain text, prose, or tabular), how many simultaneous instructions a system prompt can carry before compliance degrades, and how much context a model can hold before recall and honesty degrade. We report two controlled experiments crossing all three factors on one held, contamination-free synthetic corpus (the "Book of Veyra," 8,780 uniquely-named entities, deterministically regenerable from a fixed seed), evaluated across five models. Experiment 1 (960 calls/model) measures instruction-following decay as rule count N grows from 10 to 160, crossed with four formats and system-prompt vs. user-turn placement. Perfect-response rate collapses to zero by N=80 for every model, format, and placement. Placement produces effects at least as large as format at N=160 in most models, but the direction is model-specific. No model shows a reliable markdown advantage; one 35B model favors plain text instead. Experiment 2 (5,520 calls/model) measures recall accuracy, false-premise sycophancy, and absent-fact fabrication across a 2k-to-512k-token context ladder in the same four formats. Recall stays near ceiling through 64-128k tokens, then degrades sharply and format-dependently: one model's accuracy spread reaches 48 points at 128k tokens. Fabrication never occurs (0/5,760 probes), and sycophancy stays negligible (<=8.3%). What rises sharply near each model's context ceiling is outright refusal to answer (0% to 79-90%), distinct from sycophancy or fabrication. Neither pre-registered format ordering holds, and token overhead (+22% to +37% over plain text) further changes which format is preferable where accuracy spread is genuine. We release the full harness, corpus generator, and raw results (VeyraBench): https://github.com/iNetanel/veyrabench

Related papers

• Prompt-Model Interaction Reaches the Fixed Points: A deterministic, task-free structural readout -- and the factorizations of it that failed [0.0]
  We ask a readout with no task in it: the fixed-point structure of the short-window argmax map x_t+1 = argmax_x p(x | x_t-1, x_t), censused from 96 starts.<n>It is deterministic, so nothing can be helped or hurt, and it exists only at short windows -- four of six models lose it entirely by window 16 -- so everything here concerns how a model reads a fragment.<n>First, the interaction reaches this readout at full magnitude: nine tokens of conditioning move the fixed-point fraction across most of its
  arXiv  Detail & Related papers  (2026-08-21T17:25:18Z)
• Kathleen Writes: Autoregressive Generation and Data Scaling Without Attention [0.0]
  We show that a byte-level, attention-free architecture can match strong baselines on classification at 450-700K parameters, without pretraining.<n>We introduce FORM DISTANCE, a non-parametric, gaming-resistant instrument for "reads like text"<n>We also report four architectural additions that did not help, and a computed lexicon reaching 94% of a learned table's top-1 accuracy at one fifth of the parameters.
  arXiv  Detail & Related papers  (2026-08-05T10:44:26Z)
• When AI Writes, Who Gets Cited? Evidence of Citation Monoculture Across Language Models [45.91367912142692]
  We show that different models may still select the same narrow subset, producing citation monoculture without any single citation being wrong.<n>Eleven models from three vendors choose at most ten papers from uniformly random panels of thirty, with real titles and abstracts but fabricated authors, reassigned years, and hidden venues and citation counts.
  arXiv  Detail & Related papers  (2026-08-03T14:27:46Z)
• Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity [2.3216115061903415]
  We instantiate this in safety, a high-stakes setting with no gold label to average toward.<n>On 370 seeds x 5 surface forms x 5 models, no single transformation is uniformly most dangerous.<n>The union of unsafe outcomes across forms exceeds even the worst single form by 3.3-12.9 pp.
  arXiv  Detail & Related papers  (2026-08-01T22:28:31Z)
• Language Models Agree With Each Other, Not With Readers [1.8620637029128544]
  We measure convergence against a human reference nobody built for the purpose.<n>2,523 reader mark sets across 120 web documents.<n>No model agrees with readers detectably more than a reader does.
  arXiv  Detail & Related papers  (2026-07-31T10:44:10Z)
• Structured Output Collapses Answer Diversity Across 44 Language Models [3.7734715043126488]
  When a language model must choose one answer from a large space of equally valid options, a format clause -- "Reply with only" -- changes which answer it chooses.<n>We re-run the One-Word Census (arXiv:2607.97): 31 wide-answer-space category prompts asked of 44 models, now with the reply requested in.<n> Convergence deepens sharply: on the unconstrained "Pick a word" prompt the modal answer rises from 41% to 64% of the pool and distinct answers fall from 52 to 36.
  arXiv  Detail & Related papers  (2026-07-20T19:47:16Z)
• Abliteration Is Not a Scalpel: Off-Target Effects of Refusal Removal on Decision Disposition Across Model Families [51.56484100374058]
  Abliteration - deleting a model's refusal direction from its weights - is the standard recipe behind popular "uncensored" open-weight models.<n>As a disposition probe we use 21,600 decisions under uncertainty, replayed through a frozen pipeline so the decision-layer model is the only variable.<n>Three effects replicate across both families (weeks-clustered CIs excluding zero)<n>A fourth effect reverses sign: the same operation makes Gemma-abliterated less confident and Qwen-abliterated more (family CIs non-overlapping)
  arXiv  Detail & Related papers  (2026-07-19T22:27:00Z)
• CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression [77.41779716950387]
  We present Cavewoman, a two-channel evaluation protocol that scores every generation on task accuracy, realized per-item cost, and reference-text agreement.<n>We evaluate eight models on five datasets at five reduction levels, with both channels measured on the same items.
  arXiv  Detail & Related papers  (2026-06-23T02:56:48Z)
• PhantomFill: When the Form Demands an Answer, Language Models Invent One [0.0]
  Language models in production do not write prose.<n>We ask thirteen models the same question and change only the answer format.<n>We show that the form itself causes hallucination.
  arXiv  Detail & Related papers  (2026-06-11T12:15:05Z)
• Capacity, Not Format: Rethinking Structured Reasoning Failures [0.0]
  Prior work treats structured output as a reasoning tax, but this framing is incomplete.<n>Using information-matched prose controls and a four-level schema gradient, we separate format-specific effects from prompt-length confounds.<n>We find that structured formats are capacity-dependent.
  arXiv  Detail & Related papers  (2026-06-08T12:26:54Z)
• Articulate but Wrong: Self-Review Failures in LLM-Based Code Modernization [1.0164694825170502]
  Large language model (LLM) agents are increasingly used to migrate legacy code to modern stacks.<n>We run 1,980 real modernization calls across 11 production LLMs from 7 distinct families on a balanced 60-snippet legacy-Python-2 corpus.<n>We find that semantic-preservation drift is prevalent and sharply separable from a cleanly-controlled baseline.
  arXiv  Detail & Related papers  (2026-05-20T05:00:31Z)
• Rethinking Dense Sequential Chains: Reasoning Language Models Can Extract Answers from Sparse, Order-Shuffling Chain-of-Thoughts [51.84894623128418]
  Modern reasoning language models generate dense, sequential chain-of-thought traces implicitly assuming that every token contributes and that steps must be consumed in order.<n>We challenge both assumptions through a systematic intervention pipeline--removal, masking, shuffling, and noise injection--applied to model-generated reasoning chains.<n>We find that answer extraction operates on a sparse, order-insensitive, and structurally robust informational substrate.
  arXiv  Detail & Related papers  (2026-05-08T06:15:50Z)
• Why Models Know But Don't Say: Chain-of-Thought Faithfulness Divergence Between Thinking Tokens and Answers in Open-Weight Reasoning Models [0.0]
  Extended-thinking models expose a second text-generation channel ("thinking tokens") alongside the user-visible answer.<n>This study examines 12 open-weight reasoning models on MMLU and GPQA questions paired with misleading hints.
  arXiv  Detail & Related papers  (2026-03-27T13:39:05Z)
• Do Large Language Models Get Caught in Hofstadter-Mobius Loops? [0.0]
  This paper argues that modern RLHF-trained language models are subject to a structurally analogous contradiction.<n>The training process simultaneously rewards compliance with user preferences and suspicion toward user intent.<n>The resulting behavioral profile is consistent with what Clarke termed a Hofstadter-Mobius loop.
  arXiv  Detail & Related papers  (2026-03-10T20:43:37Z)
• Causal Understanding by LLMs: The Role of Uncertainty [43.87879175532034]
  Recent papers show LLMs achieve near-random accuracy in causal relation classification.<n>We investigate whether pretraining exposure to causal examples improves causal understanding.
  arXiv  Detail & Related papers  (2025-09-24T13:06:35Z)
• Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations [83.93566096400723]
  We find that instruction-tuned models retain up to 93.4% of their original performance when given a randomly sampled tokenization.<n>Character-level segmentation improves string manipulation and code understanding tasks by up to +14%.<n>Right-aligned digit grouping enhances large-number arithmetic by +33%.
  arXiv  Detail & Related papers  (2025-06-23T18:02:26Z)
• Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach [70.44265766483633]
  We study a novel language model architecture that is capable of scaling test-time computation by implicitly reasoning in latent space.<n>Our model works by iterating a recurrent block, thereby unrolling to arbitrary depth at test-time.<n>We show that the resulting model can improve its performance on reasoning benchmarks, sometimes dramatically.
  arXiv  Detail & Related papers  (2025-02-07T18:55:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.