What Survives When You Compress a Recursive Reasoner for the Edge?

• URL: http://arxiv.org/abs/2606.26488v1
• Date: Thu, 25 Jun 2026 00:47:15 GMT
• Title: What Survives When You Compress a Recursive Reasoner for the Edge?
• Abstract summary: Recursive reasoning models can solve complex structured tasks with only a few million parameters by repeatedly updating a latent state.<n>We find that aggressive compression preserves local prediction but destroys global reasoning.<n>We introduce carry-trajectory fidelity, similarity to the full-precision reasoning path, as a label-free signal that predicts this damage and its recovery.
• Score: 15.460381731793975
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recursive reasoning models can solve complex structured tasks with only a few million parameters by repeatedly updating a latent state. Deploying these models on edge hardware requires significant compression, but unlike conventional sequence models, quantization errors compound across recursive reasoning cycles rather than across output tokens. As a result, standard intuitions about compression fail to apply. In this work, we ask what survives when recursive reasoners are compressed. Across a full precision sweep, three tasks, and two recursive architectures, we find that aggressive compression preserves local prediction but destroys global reasoning: cell accuracy holds while puzzle-exact accuracy collapses to zero under naive INT4 pruning, distillation, and linear attention alike. Token-level objectives, including quantization-aware training, cannot repair it. The collapse is architectural -- it strikes MLP-mixing recursion but not attention on the same task -- and we reverse it with per-channel calibrated INT4 without retraining. We also introduce carry-trajectory fidelity, the cosine similarity to the full-precision reasoning path, as a label-free signal that predicts this damage and its recovery before a task evaluation. The combined result is a deployment recipe: flash-streamed embeddings remove a 99.4MB bottleneck, INT8 at one cycle matches full-depth accuracy at 6x fewer FLOPs (8MB SoC), and calibrated INT4 fits a 4MB microcontroller.

Related papers

• Steering Under Compression: Dose-Response, Capability Cost, and Failure Asymmetry in Quantized LLMs [0.0]
  Inference-time steering enables behavioral control of large language models without parameter modification.<n>Post-training quantization reduces memory and compute costs for deployment.<n>We systematically study activation steering under weight-only quantization.
  arXiv  Detail & Related papers  (2026-09-06T08:41:42Z)
• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• Compress and Forget: bitsandbytes Quantization Amplifies Proactive Interference in LLMs [0.23438564092609357]
  Proactive interference (PI) is a documented failure mode in large language models in which retrieval of a repeatedly overwritten value degrades as prior overwrites accumulate.<n>PTQ is now the default deployment path for open-weight models, yet its effect on this failure mode has not been tested.<n>We evaluate three precision levels (FP16, INT8, INT4/NF4, via bitsandbytes) across three architecturally distinct instruction-tuned models.
  arXiv  Detail & Related papers  (2026-08-19T06:17:13Z)
• KV Cache Compression Through the Lens of Transform Coding [9.884224071643606]
  We introduce Attention-Aware Transform Coding (AATC), which allocates bits over a calibration set to minimize attention-aware distortion.<n>Our method achieves near-lossless accuracy at approximately $5.8times$ compression, whereas each baseline degrades in at least some settings.
  arXiv  Detail & Related papers  (2026-08-14T11:08:01Z)
• Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery [40.94400211806987]
  We show that aggressive 2-bit inference can fail to deliver end-to-end speedup because instability in the generation process inflates total token count.<n>We analyze full reasoning traces of Qwen3 reasoning models across mathematical and commonsense benchmarks.<n>Our results show that extreme low-bit reasoning becomes practical when its failures are treated as controllable generation pathologies.
  arXiv  Detail & Related papers  (2026-06-01T10:04:09Z)
• Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution [3.6704226968275253]
  We show gradient-based methods that achieve meaningful forgetting lose it under compression, while methods that survive quantization barely change the model.<n>We present MANSU, which resolves both modes by combining causal circuit attribution to isolate the minimal forget-set subgraph.<n>We additionally introduce Circuit Attribution Divergence (CAD), a mechanistic verification metric distinguishing structural erasure from behavioral suppression.
  arXiv  Detail & Related papers  (2026-05-14T17:44:10Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• QuantSparse: Comprehensively Compressing Video Diffusion Transformer with Model Quantization and Attention Sparsification [67.15451442018258]
  Diffusion transformers exhibit remarkable video generation capability, yet their prohibitive computational and memory costs hinder practical deployment.<n>Model quantization and attention sparsification are two promising directions for compression, but each alone suffers severe performance degradation under aggressive compression.<n>We propose textbfQuantSparse, a unified framework that integrates model quantization with attention sparsification.
  arXiv  Detail & Related papers  (2025-09-28T06:49:44Z)
• COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization [8.214857267270807]
  Post-training quantization (PTQ) has emerged as a practical approach to compress large neural networks. We propose an innovative PTQ algorithm termed COMQ, which sequentially conducts coordinate-wise minimization of the layer-wise reconstruction errors. COMQ achieves remarkable results in quantizing 4-bit Vision Transformers, with a negligible loss of less than 1% in Top-1 accuracy.
  arXiv  Detail & Related papers  (2024-03-11T20:04:03Z)
• An Information Theory-inspired Strategy for Automatic Network Pruning [97.03772272417599]
  Deep convolution neural networks are well known to be compressed on devices with resource constraints.<n>Most existing network pruning methods require laborious human efforts and prohibitive computation resources.<n>We propose an information theory-inspired strategy for automatic model compression.
  arXiv  Detail & Related papers  (2021-08-19T07:03:22Z)
• AQD: Towards Accurate Fully-Quantized Object Detection [94.06347866374927]
  We propose an Accurate Quantized object Detection solution, termed AQD, to get rid of floating-point computation. Our AQD achieves comparable or even better performance compared with the full-precision counterpart under extremely low-bit schemes.
  arXiv  Detail & Related papers  (2020-07-14T09:07:29Z)
• Efficient Integer-Arithmetic-Only Convolutional Neural Networks [87.01739569518513]
  We replace conventional ReLU with Bounded ReLU and find that the decline is due to activation quantization. Our integer networks achieve equivalent performance as the corresponding FPN networks, but have only 1/4 memory cost and run 2x faster on modern GPU.
  arXiv  Detail & Related papers  (2020-06-21T08:23:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.