The Coupling Tax: How Shared Token Budgets Undermine Visible Chain-of-Thought Under Fixed Output Limits

• URL: http://arxiv.org/abs/2605.07686v1
• Date: Fri, 08 May 2026 12:54:53 GMT
• Title: The Coupling Tax: How Shared Token Budgets Undermine Visible Chain-of-Thought Under Fixed Output Limits
• Abstract summary: Chain-of-thought reasoning is often treated as a monotone way to improve language-model accuracy by letting a model think longer.<n>We identify a countervailing effect, the coupling tax: when reasoning traces and final answers share one output-token budget, long traces can crowd out the answer they are meant to support.<n>Across GSM8K, MATH-500, and five BIG-Bench Hard tasks with Qwen3 models at three scales, non-thinking mode matches or outperforms thinking mode on GSM8K and MATH-500 at every budget up to 2048 tokens.
• Score: 12.316478346055618
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Chain-of-thought reasoning is often treated as a monotone way to improve language-model accuracy by letting a model think longer. We identify a countervailing effect, the coupling tax: when reasoning traces and final answers share one output-token budget, long traces can crowd out the answer they are meant to support. Across GSM8K, MATH-500, and five BIG-Bench Hard tasks with Qwen3 models at three scales, non-thinking mode matches or outperforms thinking mode on GSM8K and MATH-500 at every budget up to 2048 tokens, while harder tasks shift the crossover to larger budgets. We derive a truncation-waste decomposition, $\mathrm{Acc}_{\mathrm{think}}(b)=α_c F_L(b)+α_t(1-F_L(b))$, that predicts this crossover from chain-length and accuracy statistics and explains inverse scaling within the Qwen family. A DeepSeek-R1-Distill-Llama-8B replication shows the same pattern under a different thinking interface. As a mitigation, split-budget generation decouples reasoning and answer budgets; on full MATH-500, IRIS reaches 74.0% accuracy, a strengthened extraction variant reaches 78.8%, and a fixed non-oracle SC+IRIS gate reaches 83.6%. The results show that test-time reasoning should be evaluated as a budget-allocation problem, not only as a question of whether longer traces are available.

Related papers

• Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation [5.154153910688557]
  We study whether a model can learn to allocate its own reasoning effort by choosing one of three modes.<n>The choice is learned inside Group Relative Policy Optimization with no separate router.<n>We build a reasoning model that adaptively chooses how much to reason for each problem.
  arXiv  Detail & Related papers  (2026-08-20T16:54:08Z)
• ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling [60.48279351544583]
  ThinkRetrieve is a test-time scaling framework for large reasoning models.<n>It retrieves relevant exemplars at each intermediate step and injects them directly into the thinking trace.<n>It consistently improves accuracy over standard test-time scaling.
  arXiv  Detail & Related papers  (2026-08-11T13:58:07Z)
• Stateful Reasoning via Insight Replay [51.85629502016196]
  Chain-of-Thought (CoT) reasoning has become a foundation for eliciting multi-step reasoning in large language models.<n>We identify a major cause of this phenomenon: as the CoT grows, the model's attention to critical insights produced earlier in the trace gradually weakens.<n>We propose textbfInsightReplay, a stateful reasoning approach in which the model periodically extracts critical insights from its reasoning trace and replays them near the active generation frontier.
  arXiv  Detail & Related papers  (2026-05-14T06:52:59Z)
• How Much Thinking is Enough? Quantifying and Understanding Redundancy in LLM Reasoning [11.159657169036564]
  Reasoning-capable large language models solve hard problems by emitting long chains of thought.<n>But how much of this deliberation is actually necessary has never been measured at scale or explained from first principles.<n>We formalise reasoning redundancy directly in terms of the reasoning model itself.
  arXiv  Detail & Related papers  (2026-04-21T05:32:02Z)
• SyncThink: A Training-Free Strategy to Align Inference Termination with Reasoning Saturation [11.021989271617835]
  We present SyncThink, a training-free and plug-and-play decoding method that reduces Chain-of-Thought overhead without modifying model weights.<n>We find that answer tokens attend weakly to early reasoning and instead focus on the special token "/think", indicating an information bottleneck.<n>Experiments on GSM8K, MMLU, GPQA, and BBH across three DeepSeek-R1 distilled models show that SyncThink achieves 62.00 percent average Top-1 accuracy.
  arXiv  Detail & Related papers  (2026-01-07T07:00:15Z)
• Complexity Agnostic Recursive Decomposition of Thoughts [3.3864434164156934]
  We introduce CARD (Complexity Agnostic Recursive Decomposition), a framework that predicts problem complexity before generation and adapts decomposition accordingly.<n> CARD achieves 81.4% to 89.2% accuracy on GSM8K while reducing token cost by 1.88x to 2.40x compared to fixed decomposition baselines.
  arXiv  Detail & Related papers  (2025-12-10T06:03:42Z)
• Think Right: Learning to Mitigate Under-Over Thinking via Adaptive, Attentive Compression [68.69801176669843]
  We propose an online post-training RL method that prunes redundant steps and estimates difficulty.<n> TRAAC (Think Right with Adaptive, Attentive Compression) achieves an average absolute accuracy gain of 8.4%.<n>Although our models are trained on math datasets, they show accuracy and efficiency gains on out-of-distribution non-math datasets.
  arXiv  Detail & Related papers  (2025-10-02T02:00:20Z)
• VeriThinker: Learning to Verify Makes Reasoning Model Efficient [52.74493506816969]
  Large Reasoning Models excel at complex tasks using Chain-of-Thought (CoT) reasoning.<n>Their tendency to overthinking leads to unnecessarily lengthy reasoning chains.<n>We introduce VeriThinker, a novel approach for CoT compression.
  arXiv  Detail & Related papers  (2025-05-23T14:17:56Z)
• Computational-Statistical Tradeoffs at the Next-Token Prediction Barrier: Autoregressive and Imitation Learning under Misspecification [50.717692060500696]
  Next-token prediction with the logarithmic loss is a cornerstone of autoregressive sequence modeling.<n>Next-token prediction can be made robust so as to achieve $C=tilde O(H)$, representing moderate error amplification.<n>No computationally efficient algorithm can achieve sub-polynomial approximation factor $C=e(log H)1-Omega(1)$.
  arXiv  Detail & Related papers  (2025-02-18T02:52:00Z)
• Think Smarter not Harder: Adaptive Reasoning with Inference Aware Optimization [48.5702580429946]
  In this work, we propose a way to allow models to be aware of inference budgets by formulating it as utility with respect to an inference budget constraint.<n>In a nutshell, models fine-tuned through IBPO learn to understand'' the difficulty of queries and allocate inference budgets to harder ones.<n>These improvements are approximately $2$x those of self-consistency under the same budgets.
  arXiv  Detail & Related papers  (2025-01-29T20:20:48Z)
• DCR: Divide-and-Conquer Reasoning for Multi-choice Question Answering with LLMs [9.561022942046279]
  We propose Divide and Conquer Reasoning (DCR) to enhance the reasoning capability of large language models (LLMs) We first categorize questions into two subsets based on confidence score ($mathcalCS$), which is estimated by statistical frequency of generated answers. In particular, we first categorize questions into two subsets based on confidence score ($mathcalCS$), which is estimated by statistical frequency of generated answers.
  arXiv  Detail & Related papers  (2024-01-10T14:38:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.