Calibrated e-CUSUM Decoding for Quantized Reasoning Models: Why Token Log-Probability Is the Wrong Observable for Decoding Monitors

• URL: http://arxiv.org/abs/2607.11317v1
• Date: Mon, 13 Jul 2026 09:34:36 GMT
• Title: Calibrated e-CUSUM Decoding for Quantized Reasoning Models: Why Token Log-Probability Is the Wrong Observable for Decoding Monitors
• Abstract summary: Low-bit quantization makes small reasoning models inexpensive to deploy but can degrade their chains of thought.<n>We show that the centered token log-probability increment $log p(w_t)+H_t$ is the wrong observable for this purpose.<n>We introduce a training-free decoding controller that combines a degeneration-aware alarm score fusing token uncertainty with explicit repetition.
• Score: 0.6999740786886536
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Low-bit quantization makes small reasoning models inexpensive to deploy but can degrade their chains of thought. This motivates decoder-side monitors that intervene when generation becomes unreliable. We show that a natural candidate, the centered token log-probability increment $\log p(w_t)+H_t$, is the wrong observable for this purpose. Under the model's own sampling law it is a mean-zero martingale by construction, so it measures sampling self-consistency rather than trajectory health and is nearly silent during confident repetition, where both $\log p(w_t)$ and entropy are close to zero. We introduce a training-free decoding controller that combines (i) a degeneration-aware alarm score fusing token uncertainty with explicit verbatim repetition and (ii) a calibrated e-process-inspired sequential detector. The raw product process is Ville-valid under a conditional-mean null, while the deployed CUSUM-floored statistic is treated as an empirical change detector because the score is history-dependent and autocorrelated. On GSM8K with DeepSeek-R1-Distill-Qwen-1.5B in FP16 and INT4, calibration turns a monitor that fires on 93--95% of generations into a selective detector of failing traces ($φ\approx 0.3$, precision $\approx 0.6$ against a 0.38 base rate). In this pilot, the controller reduces measured verbatim-degeneration signals and yields a positive but statistically inconclusive INT4 accuracy change from 63% to 69% (paired McNemar $p=0.18$, $n=100$), at a 28% token-budget cost. We also find that non-termination, rather than looping, is the dominant failure mode on GSM8K. The main contribution is methodological: an explanation of why centered token log-probability is inadequate for decoder monitoring and a calibrated, cautiously evaluated replacement.

Related papers

• Stateful CARS: Exact Cross-History Reuse for Policy-Constrained LLM Agents [0.9558392439655014]
  We study exact sampling from the model distribution conditioned on a hard stateful validator.<n>Stateful CARS freezes a bank of sound state--continuation schemas within each attempt.<n>Cross-history transfer helps only in an internal matched-key ablation.
  arXiv  Detail & Related papers  (2026-08-08T18:26:25Z)
• Cybersecurity Detection Classification with Reasoning-enabled Language Models [50.72675435043546]
  A major issue in Security Operations Centers (SOCs) is alert fatigue.<n>Prior work prompts or fine-tunes large language models (LLMs) to emit a triage label directly, but does not train them to reason about whether a detection is a genuine threat.<n>We train a chain-of-thought (CoT) reasoning-enabled triage classifier on real, human-labeled Windows endpoint detections.
  arXiv  Detail & Related papers  (2026-07-30T16:22:13Z)
• Mean-to-Score Discrete Diffusion: Posterior-Mean Denoisers for Score Entropy [17.48959208263029]
  We introduce emphmean-to-score (M2S), which predicts a clean-token posterior mean and converts it to the score through an exact kernel-dependent linear map.<n>For uniform corruption, it maps the probability simplex onto the bridge polytope; for absorbing-mask corruption, the resulting objective recovers MD4 exactly.
  arXiv  Detail & Related papers  (2026-07-23T14:37:30Z)
• When Agents Commit Too Soon: Diagnosing Premature Commitment in LLM Agents [0.0]
  Long-answer LLM agents can fail quietly: they settle on one reading of the evidence early, then spend the rest of the run defending it.<n>We define representational commitment as cross-run hidden-state convergence at a fixed reasoning step.<n>A runtime monitor detects inconsistent trajectories from hidden states at AUROC up to 0.97 (0.85--0.88 under a stricter split)
  arXiv  Detail & Related papers  (2026-06-22T07:13:13Z)
• Stop Early, Spend Less: Hidden-State Probes as a Practical Recipe for Streaming Moderation of LLM Outputs [0.0]
  Existing approaches rely on a separate moderation model applied after generation.<n>We observe that the signal needed for moderation is already present in the model hidden states.<n>We train lightweight token-level probes that operate directly on internal activations.
  arXiv  Detail & Related papers  (2026-06-09T07:01:43Z)
• Lost in Fog: Sensor Perturbations Expose Reasoning Fragility in Driving VLAs [0.0]
  We present a perturbation study of Vision-Language-Action (VLA) robustness in autonomous driving.<n>We find that reasoning consistency is a high-fidelity indicator of trajectory reliability.
  arXiv  Detail & Related papers  (2026-05-20T17:34:02Z)
• The Verification Tax: Fundamental Limits of AI Auditing in the Rare-Error Regime [0.0]
  Most cited calibration result in deep learning -- post-temperature-scaling ECE of 0.012 on CIFAR-100 -- is below the statistical noise floor.<n>We prove this is not a failure of the experiment but a law: the minimax rate for estimating calibration error with model error rate epsilon is Theta((Lepsilon/m)2/3), and no estimator can beat it.
  arXiv  Detail & Related papers  (2026-04-14T16:48:24Z)
• Epistemic Observability in Language Models [0.0]
  We find that models report highest confidence precisely when they are fabricating.<n>We prove, under formal assumptions, that this is not a capability gap but an observational one.<n>We construct a tensor interface that escapes the impossibility by exporting computational byproducts.
  arXiv  Detail & Related papers  (2026-03-20T21:59:34Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning [80.104336426172]
  Chain-of-thought (CoT) enhances problem-solving ability of large language models.<n>CoT incurs substantial inference cost due to long autoregressive trajectories.<n>We introduce R-Stitch, a training-free hybrid decoding framework.
  arXiv  Detail & Related papers  (2025-07-23T08:14:36Z)
• VeriThinker: Learning to Verify Makes Reasoning Model Efficient [52.74493506816969]
  Large Reasoning Models excel at complex tasks using Chain-of-Thought (CoT) reasoning.<n>Their tendency to overthinking leads to unnecessarily lengthy reasoning chains.<n>We introduce VeriThinker, a novel approach for CoT compression.
  arXiv  Detail & Related papers  (2025-05-23T14:17:56Z)
• Computational-Statistical Tradeoffs at the Next-Token Prediction Barrier: Autoregressive and Imitation Learning under Misspecification [50.717692060500696]
  Next-token prediction with the logarithmic loss is a cornerstone of autoregressive sequence modeling.<n>Next-token prediction can be made robust so as to achieve $C=tilde O(H)$, representing moderate error amplification.<n>No computationally efficient algorithm can achieve sub-polynomial approximation factor $C=e(log H)1-Omega(1)$.
  arXiv  Detail & Related papers  (2025-02-18T02:52:00Z)
• Towards Optimal Statistical Watermarking [95.46650092476372]
  We study statistical watermarking by formulating it as a hypothesis testing problem. Key to our formulation is a coupling of the output tokens and the rejection region. We characterize the Uniformly Most Powerful (UMP) watermark in the general hypothesis testing setting.
  arXiv  Detail & Related papers  (2023-12-13T06:57:00Z)
• Probing Model Signal-Awareness via Prediction-Preserving Input Minimization [67.62847721118142]
  We evaluate models' ability to capture the correct vulnerability signals to produce their predictions. We measure the signal awareness of models using a new metric we propose- Signal-aware Recall (SAR) The results show a sharp drop in the model's Recall from the high 90s to sub-60s with the new metric.
  arXiv  Detail & Related papers  (2020-11-25T20:05:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.