Structured Output Collapses Answer Diversity Across 44 Language Models

• URL: http://arxiv.org/abs/2607.18476v1
• Date: Mon, 20 Jul 2026 19:47:16 GMT
• Title: Structured Output Collapses Answer Diversity Across 44 Language Models
• Abstract summary: When a language model must choose one answer from a large space of equally valid options, a format clause -- "Reply with only" -- changes which answer it chooses.<n>We re-run the One-Word Census (arXiv:2607.97): 31 wide-answer-space category prompts asked of 44 models, now with the reply requested in.<n> Convergence deepens sharply: on the unconstrained "Pick a word" prompt the modal answer rises from 41% to 64% of the pool and distinct answers fall from 52 to 36.
• Score: 3.7734715043126488
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When a language model must choose one answer from a large space of equally valid options, a format clause -- "Reply with JSON only" -- changes which answer it chooses. We re-run the One-Word Census (arXiv:2607.12796): 31 wide-answer-space category prompts asked of 44 models, now with the reply requested in JSON -- no schema enforcement, no constrained decoding, only the request. Convergence deepens sharply: on the unconstrained "Pick a word" prompt the modal answer rises from 41% to 64% of the pool and distinct answers fall from 52 to 36; mean answer-choice surprisal drops from 1.80 to 1.58 bits. The tax is progressive: six of 44 models move individually (BH-FDR q=.10), all toward the mode, led by the most distinctive models, while the conformist floor is immobile. It is a sharpener, not a re-indexer -- the plain-chat modal answer survives in 28 of 31 categories. Defaults are register-indexed: a within-run re-sample (n=20) finds JSON shifts 53% of a model's stable chat defaults, mostly back to the crowd, and installs defaults absent from chat (Claude Fable 5 answers "cerulean" for colour 0% of the time in chat, 100% in JSON). Full-battery controls reveal a register gradient: compression is significant and specific to the answer-delivery formats models are trained to speak (JSON -0.22 bits, p=.0002; XML -0.19, p=.002), absent for YAML and CSV, and reversed for an arbitrary bracket wrapper (+0.13, p=.009) -- weighing the mechanism toward tool-use post-training. Enforcing the schema at the decoder (response_format) compresses no further than the request (-0.03 bits): the collapse lives in the model's response to the register, not the decoder. Structured output is how software consumes language models, and that surface is served by a measurably more homogeneous model than the chat surface on which models are evaluated, compared, and chosen.

Related papers

• Knowing When Not to Answer: Pseudo-Ensembles for Abstention in Music Audio-Language Models [3.6463026920885824]
  Music audio-language models are evaluated almost entirely by accuracy on multiple-choice questions.<n>What is missing is a way to tell when the model does not know the answer, so that it can abstain instead of guessing.<n>We build pseudo-ensembles from one pretrained model by perturbing its input in ways that cannot change the correct answer.
  arXiv  Detail & Related papers  (2026-09-03T18:21:25Z)
• Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models [0.0]
  Two experiments cross all three factors on one held, contamination-free synthetic corpus.<n>Experiment 1 measures instruction-following decay as rule count N grows from 10 to 160, crossed with four formats and system-prompt vs. user-turn placement.<n>Experiment 2 measures recall accuracy, false-premise sycophancy, and absent-fact fabrication across a 2k-to-512k-token context ladder.
  arXiv  Detail & Related papers  (2026-07-21T16:31:35Z)
• The One-Word Census: Answer-Choice Conformity Across 44 Language Models [3.7734715043126488]
  We characterize convergence of 31 single-turn prompts, each naming a category with many valid one-word answers.<n>We score each model by answer-choice surprisal: the average $-log2$ probability of its answers under the pooled answers of all other models.<n> Persona- and community-tuned models are the most divergent; the newest mainline flagships are the most conformist.
  arXiv  Detail & Related papers  (2026-07-14T14:12:05Z)
• CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression [77.41779716950387]
  We present Cavewoman, a two-channel evaluation protocol that scores every generation on task accuracy, realized per-item cost, and reference-text agreement.<n>We evaluate eight models on five datasets at five reduction levels, with both channels measured on the same items.
  arXiv  Detail & Related papers  (2026-06-23T02:56:48Z)
• PhantomFill: When the Form Demands an Answer, Language Models Invent One [0.0]
  Language models in production do not write prose.<n>We ask thirteen models the same question and change only the answer format.<n>We show that the form itself causes hallucination.
  arXiv  Detail & Related papers  (2026-06-11T12:15:05Z)
• Paraphrase-Induced Output-Mode Collapse: When LLMs Break Character Under Semantically Equivalent Inputs [0.9525172018746524]
  We observe a systematic failure mode we call prompt-variant output-mode collapse.<n>When a closed-form prompt asks for a bare label or a single choice token, content-preserving prompt variants can push the model into conversational prose.<n>We release PARACONSIST, a benchmark of 150 base queries with five lexical, syntactic, and semantic-expansion prompt variants each.
  arXiv  Detail & Related papers  (2026-05-06T09:11:10Z)
• Micro Language Models Enable Instant Responses [49.192613030724424]
  Edge devices such as smartwatches and smart glasses cannot continuously run even the smallest 100M-1B parameter language models due to power and compute constraints.<n>We introduce micro language models ($$LMs) that instantly generate the first 4-8 words of a contextually grounded response on-device.<n>We show that useful language generation survives at this extreme scale with our models matching several 70M-256M-class existing models.
  arXiv  Detail & Related papers  (2026-04-21T16:31:12Z)
• HuggingR$^{4}$: A Progressive Reasoning Framework for Discovering Optimal Model Companions [50.61510609116118]
  HuggingR$4$ is a novel framework that combines Reasoning, Retrieval, Refinement, and Reflection to efficiently select models.<n>It attains a workability rate of 92.03% and a reasonability rate of 82.46%, surpassing existing method by 26.51% and 33.25% respectively.
  arXiv  Detail & Related papers  (2025-11-24T03:13:45Z)
• Measuring short-form factuality in large language models [50.15055025275888]
  We present SimpleQA, a benchmark that evaluates the ability of language models to answer short, fact-seeking questions. SimpleQA is adversarially collected against GPT-4 responses. Each answer in SimpleQA is graded as either correct, incorrect, or not attempted.
  arXiv  Detail & Related papers  (2024-11-07T01:58:42Z)
• Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questions [103.20281438405111]
  Multiple-choice question answering (MCQA) is a key competence of performant transformer language models.<n>We employ vocabulary projection and activation patching methods to localize key hidden states that encode relevant information for predicting the correct answer.<n>We show that subsequent layers increase the probability of the predicted answer symbol in vocabulary space, and that this probability increase is associated with a sparse set of attention heads with unique roles.
  arXiv  Detail & Related papers  (2024-07-21T00:10:23Z)
• "My Answer is C": First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models [40.867655189493924]
  Open-ended nature of language generation makes evaluation of large language models (LLMs) challenging. One common evaluation approach uses multiple-choice questions (MCQ) to limit the response space. We evaluate how aligned first-token evaluation is with the text output along several dimensions.
  arXiv  Detail & Related papers  (2024-02-22T12:47:33Z)
• Zero-Shot Dialogue Disentanglement by Self-Supervised Entangled Response Selection [79.37200787463917]
  Dialogue disentanglement aims to group utterances in a long and multi-participant dialogue into threads. This is useful for discourse analysis and downstream applications such as dialogue response selection. We are the first to propose atextbfzero-shot dialogue disentanglement solution.
  arXiv  Detail & Related papers  (2021-10-25T05:15:01Z)
• Investigation of Sentiment Controllable Chatbot [50.34061353512263]
  In this paper, we investigate four models to scale or adjust the sentiment of the response. The models are a persona-based model, reinforcement learning, a plug and play model, and CycleGAN. We develop machine-evaluated metrics to estimate whether the responses are reasonable given the input.
  arXiv  Detail & Related papers  (2020-07-11T16:04:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.