When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation

• URL: http://arxiv.org/abs/2606.21704v1
• Date: Fri, 19 Jun 2026 19:31:37 GMT
• Title: When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation
• Abstract summary: Chain-of-Thought (CoT) distillation transfers multi-step reasoning from large reasoning models to smaller students.<n>Existing CoT compression methods fall into two families, selective pruning and generative rewriting.<n>We recast CoT compression along three dimensions: importance criterion, restructuring level, and compression budget.
• Score: 22.684181578779132
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Chain-of-Thought (CoT) distillation transfers multi-step reasoning from large reasoning models to smaller students, but verbose teacher traces inflate both training and inference cost. Existing CoT compression methods fall into two families, selective pruning and generative rewriting, yet prior studies have left key factors entangled: granularity is confounded with importance criteria in pruning, restructuring level is rarely isolated in rewriting, and compression budgets are not systematically evaluated across domains or regimes. We recast CoT compression along three dimensions: importance criterion, restructuring level, and compression budget. Sweeping these across two model families, Math and General domains, and Long-/Short-CoT regimes, we find that (i) importance criterion utility is strictly governed by granularity: step-level criteria converge on a shared reasoning backbone, while token-level pruning requires symbol-aware signals to preserve the logical core; (ii) restructuring level inverts across domains: Math degrades monotonically with structural disruption, while aggressive rewriting acts as a denoiser on General tasks; (iii) training-time compression does not necessarily translate to inference-time savings: Long-CoT students retain verbose habits despite concise supervision, making the training ratio an optimistic lower bound on deployment cost. These findings yield condition-aware guidelines for matching compression to deployment context.

Related papers

• Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation [60.55792673956761]
  We propose textbfRubric-Conditioned Self-Distillation, a framework that incorporates rubrics as structured, fine-grained feedback for on-policy self-distillation.<n>We evaluate on a diverse suite of science reasoning benchmarks and results show that rubric-conditioned self-distillation effectively converts rubric-level criteria into token-level guidance.
  arXiv  Detail & Related papers  (2026-06-17T17:54:04Z)
• SLAT: Segment-Level Adaptive Trimming for Efficient CoT Reasoning [26.883032881561423]
  We propose textscSLAT (Segment-Level Adaptive Trimming), an RL framework that selectively suppresses redundant segments.<n> Empirical results indicate that textscSLAT establishes a superior accuracy-efficiency frontier, reducing reasoning length by $50%$.<n>Overall, our results suggest that theoretically grounded, segment-aware trimming is a promising direction for efficient CoT reasoning in large language models.
  arXiv  Detail & Related papers  (2026-05-29T04:37:49Z)
• A Geometric View of SRC: Learning Representations for Stable Residual Inference [0.0]
  Reconstruction-based inference assigns a class by comparing class-wise reconstruction residuals.<n>Sparse Representation Classification (SRC) is a canonical instance whose reliability depends on the geometry of the learned representation.<n>We formalize residual-ordering stability through a residual margin and characterize geometric obstructions.
  arXiv  Detail & Related papers  (2026-05-28T09:33:18Z)
• Structural Rationale Distillation via Reasoning Space Compression [34.91106623292321]
  Reasoning Path Compression constrains the teacher to follow a compact, dynamically maintained bank of reusable high-level reasoning paths.<n>For each training question, D-RPC retrieves the most relevant path and conditions the teacher to follow it, producing rationales consistent across similar problems yet diverse enough to cover different problem types.
  arXiv  Detail & Related papers  (2026-05-08T02:15:52Z)
• Constraint-Rectified Training for Efficient Chain-of-Thought [60.52883907721588]
  Chain-of-Thought (CoT) has significantly enhanced the reasoning capabilities of Large Language Models (LLMs)<n>While longer reasoning traces can improve answer quality and unlock abilities such as self-correction, they also incur high inference costs and often introduce redundant steps, known as overthinking.<n>Recent research seeks to develop efficient reasoning strategies that balance reasoning length and accuracy.
  arXiv  Detail & Related papers  (2026-02-13T02:13:45Z)
• Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression [55.63153956934198]
  Chain-of-Thought (CoT) reasoning successfully enhances the reasoning capabilities of Large Language Models (LLMs)<n>Existing CoT compression methods often suffer from a critical loss of logical fidelity at high compression ratios.<n>We propose a novel EXTreme-RAtio Chain-of-Thought Compression framework, termed Extra-CoT, which aggressively reduces the token budget while preserving answer accuracy.
  arXiv  Detail & Related papers  (2026-02-09T06:57:15Z)
• Short Chains, Deep Thoughts: Balancing Reasoning Efficiency and Intra-Segment Capability via Split-Merge Optimization [68.89915707647138]
  Large Reasoning Models (LRMs) have demonstrated impressive capabilities in solving complex tasks through the generation of long reasoning chains.<n>We propose textbfCoSMo (textbfSplit-textbfMerge textbfOptimization), a framework designed to eliminate structural redundancy rather than indiscriminately restricting token volume.
  arXiv  Detail & Related papers  (2026-02-03T05:54:28Z)
• CtrlCoT: Dual-Granularity Chain-of-Thought Compression for Controllable Reasoning [29.057579417751203]
  Chain-of-thought (CoT) prompting improves LLM reasoning but incurs high latency and memory cost due to verbose traces.<n>We propose textbfCtrlCoT, a dual-granularity CoT compression framework that harmonizes semantic abstraction and token-level pruning.
  arXiv  Detail & Related papers  (2026-01-28T10:38:49Z)
• Structured Reasoning for Large Language Models [59.215789462977206]
  We propose Structured Reasoning (SCR), a framework that decouples reasoning trajectories into explicit, evaluable, and trainable components.<n>SCR substantially improves reasoning efficiency and self-verification.<n>Compared with existing reasoning paradigms, it reduces output token length by up to 50%.
  arXiv  Detail & Related papers  (2026-01-12T04:04:01Z)
• Provable Benefit of Curriculum in Transformer Tree-Reasoning Post-Training [76.12556589212666]
  We show that curriculum post-training avoids the exponential complexity bottleneck.<n>Under outcome-only reward signals, reinforcement learning finetuning achieves high accuracy with sample complexity.<n>We establish guarantees for test-time scaling, where curriculum-aware querying reduces both reward oracle calls and sampling cost from exponential to order.
  arXiv  Detail & Related papers  (2025-11-10T18:29:54Z)
• Large Reasoning Models Learn Better Alignment from Flawed Thinking [56.08883934423522]
  Large reasoning models (LRMs) "think" by generating structured chain-of-thought (CoT) before producing a final answer.<n>We propose RECAP, a principled reinforcement learning (RL) method for post-training that explicitly teaches models to override flawed reasoning trajectories.
  arXiv  Detail & Related papers  (2025-10-01T14:15:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.