Capacity, Not Format: Rethinking Structured Reasoning Failures

• URL: http://arxiv.org/abs/2606.09410v1
• Date: Mon, 08 Jun 2026 12:26:54 GMT
• Title: Capacity, Not Format: Rethinking Structured Reasoning Failures
• Abstract summary: Prior work treats structured output as a reasoning tax, but this framing is incomplete.<n>Using information-matched prose controls and a four-level schema gradient, we separate format-specific effects from prompt-length confounds.<n>We find that structured formats are capacity-dependent.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prior work treats structured output as a reasoning tax, but this framing is incomplete: the cost of formatting depends strongly on a model's spare capacity. Using information-matched prose controls and a four-level schema complexity gradient, we separate format-specific effects from prompt-length confounds across 4 models and 5 benchmarks with 0% parse failures on successfully generated responses. We find that structured formats are capacity-dependent. Models with sufficient headroom absorb JSON constraints without degradation (Sonnet: $88.7\pm4.0$% JSON vs. $89.3\pm1.7$% CoT on MATH-Hard). In contrast, formats severely degrade models operating near their limits through two distinct mechanisms. First, under standard token budgets, Haiku drops 36.2pp ($p < 0.0001$) largely due to truncation. Second, even with extended budgets eliminating truncation, GPT-4o-mini drops 28.0pp ($p < 0.001$), revealing pure capacity competition independent of token exhaustion. This format penalty scales with schema complexity (McNemar $p < 0.0001$) and cannot be explained by prompt length alone. Furthermore, these results qualify claims of frontier model immunity: on AIME competition math, Opus 4.7 drops from 96.2% to 91.0% under JSON ($-5.3$pp; the displayed percentages are independently rounded, exact difference is $7/133 = 5.26$pp $\approx 5.3$pp). A delayed-structure ablation -- reasoning freely before formatting -- recovers most of the lost accuracy (3-run mean: 80--87%), supporting the capacity competition mechanism. The practical implication is not to avoid structured output, but to match it to capacity: when a model is near its limits, think first, format later.

Related papers

• Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models [0.0]
  Two experiments cross all three factors on one held, contamination-free synthetic corpus.<n>Experiment 1 measures instruction-following decay as rule count N grows from 10 to 160, crossed with four formats and system-prompt vs. user-turn placement.<n>Experiment 2 measures recall accuracy, false-premise sycophancy, and absent-fact fabrication across a 2k-to-512k-token context ladder.
  arXiv  Detail & Related papers  (2026-07-21T16:31:35Z)
• Scaffolding the Strategist: Architecture-Dependent Reasoning Interventions in Hotelling Spatial Markets [0.01269104766024433]
  We investigate whether structured reasoning interventions improve the strategic economic reasoning of large language models.<n>We evaluate GPT-4.1-mini (a standard instruction-following model) and GPT-5-mini (a reasoning-optimized model) under five conditions.<n>We find a statistically significant crossover interaction between scaffolding type and model architecture.
  arXiv  Detail & Related papers  (2026-07-03T10:23:31Z)
• Cluster, Route, Escalate: Cascaded Framework for Cost-Aware LLM Serving [69.5155152169752]
  We propose a two-stage cascaded solution for large language models (LLMs)<n>Stage 1 clusters incoming queries and assigns each cluster to its most cost-effective model.<n>Stage 2 adds a quality estimation (QE) cascade; when an output from Stage 1 is judged low-quality, the query is escalated to a stronger model.<n>On the test datasets, the cascaded system retains 97-99% of the strongest model's accuracy while reducing Time Per Output Token (TPOT)<n>It requires only task-correctness labels and adapts to changes in the model pool without manual reconfiguration.
  arXiv  Detail & Related papers  (2026-06-25T18:29:24Z)
• CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression [77.41779716950387]
  We present Cavewoman, a two-channel evaluation protocol that scores every generation on task accuracy, realized per-item cost, and reference-text agreement.<n>We evaluate eight models on five datasets at five reduction levels, with both channels measured on the same items.
  arXiv  Detail & Related papers  (2026-06-23T02:56:48Z)
• Models Can Model, But Can't Bind: Structured Grounding in Text-to-Optimization [54.749573452394664]
  We find that accuracy collapses as instance data grows, even when the formulation itself is simple.<n>We address this via a simple inference-time approach, BIND, which externalizes numeric data to structured files so the model binds data rather than from prompt prompt.<n>We validate our hypothesis by finetuning a model exclusively on binding and show that it outperforms end-to-end SFT and RL across three structurally distinct optimization categories.
  arXiv  Detail & Related papers  (2026-05-20T21:25:41Z)
• The Constraint Tax: Measuring Validity-Correctness Tradeoffs in Structured Outputs for Small Language Models [0.5482532589225553]
  This paper targets on-device and low-cost small language model (SLM) deployments.<n>We show that hard output constraints improve reliability without changing the underlying answer.<n>We introduce emphconstraint tax, a measurement protocol for isolating the answer and executable-accuracy loss.
  arXiv  Detail & Related papers  (2026-05-20T07:11:32Z)
• One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulness [12.183451602438753]
  We show that simple lexical constraints (banning a single punctuation character or common word) cause instruction-tuned LLMs to collapse their responses.<n>We show that base models show no systematic collapse under identical constraints, with effects that are small, noisy, and bidirectional.
  arXiv  Detail & Related papers  (2026-04-14T17:40:01Z)
• The Format Tax [30.128781228264216]
  We find that structured output requirements substantially degrade reasoning and writing performance across open-weight models.<n>This diagnosis points to a simple principle: decouple reasoning from formats.<n>Most recent closed-weight models show little to no format tax, suggesting the problem is not inherent to structured generation but a gap that current open-weight models have yet to close.
  arXiv  Detail & Related papers  (2026-04-04T07:16:28Z)
• Did You Forget What I Asked? Prospective Memory Failures in Large Language Models [0.0]
  Large language models often fail to satisfy formatting instructions when they must simultaneously perform demanding tasks.<n>We study this behaviour through a prospective memory inspired lens from cognitive psychology, using a controlled paradigm.<n>Vulnerability is highly type-dependent, with drops up to 50%, while avoidance constraints remain comparatively robust.<n>A salience-enhanced format (explicit instruction framing plus a trailing reminder) recovers much of the lost compliance, restoring performance to 90-100% in many settings.
  arXiv  Detail & Related papers  (2026-03-07T05:58:19Z)
• PromptPort: A Reliability Layer for Cross-Model Structured Extraction [0.11280931253550518]
  Structured extraction with LLMs fails in production not because models lack understanding, but because output formatting is unreliable across models and prompts.<n>We present PromptPort, a reliability layer combining deterministic canonicalization with a lightweight verifier (DistilBERT) and a safe-override policy.<n>The method generalizes to held-out model families and provides explicit abstention when uncertain, enabling reliable structured extraction in production deployments.
  arXiv  Detail & Related papers  (2026-01-06T03:54:27Z)
• Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Space [56.37266873329401]
  Large Language Models (LLMs) apply uniform computation to all tokens, despite language exhibiting highly non-uniform information density.<n>We propose $textbfDynamic Large Concept Models (DLCM)$, a hierarchical language modeling framework that learns semantic boundaries from latent representations and shifts from tokens to a compressed concept space where reasoning is more efficient.
  arXiv  Detail & Related papers  (2025-12-31T04:19:33Z)
• Catch Your Breath: Adaptive Computation for Self-Paced Sequence Production [55.76222360698305]
  We explore a class of supervised training objectives that allow a language model to dynamically and autonomously scale the number of compute steps used for each input token.<n>For any token, the model can request additional compute steps by emitting a don't know> output.<n>We find that the CYB model requests additional steps when doing so improves accuracy, and the model adapts its processing time to token-level complexity and context.
  arXiv  Detail & Related papers  (2025-10-13T21:07:05Z)
• DISCO: Diversifying Sample Condensation for Efficient Model Evaluation [59.01400190971061]
  Costly evaluation reduces inclusivity, slows the cycle of innovation, and worsens environmental impact.<n>We argue that promoting diversity among samples is not essential; what matters is to select samples thatmaximise diversity in model responses.<n>Our method, $textbfDiversifying Sample Condensation (DISCO)$, selects the top-k samples with the greatest model disagreements.
  arXiv  Detail & Related papers  (2025-10-09T08:53:59Z)
• Gradient-Free Structured Pruning with Unlabeled Data [57.999191898036706]
  We propose a gradient-free structured pruning framework that uses only unlabeled data. Up to 40% of the original FLOP count can be reduced with less than a 4% accuracy loss across all tasks considered.
  arXiv  Detail & Related papers  (2023-03-07T19:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.