BitCal-TTS: Bit-Calibrated Test-Time Scaling for Quantized Reasoning Models

• URL: http://arxiv.org/abs/2605.05561v1
• Date: Thu, 07 May 2026 01:10:34 GMT
• Title: BitCal-TTS: Bit-Calibrated Test-Time Scaling for Quantized Reasoning Models
• Abstract summary: BitCal-TTS is a lightweight runtime controller for greedy 4-bit inference.<n>It integrates with standard Hugging Face 4-bit inference using forward hooks for logits and last-layer hidden states.<n>On evaluation shards of GSM8K with Qwen2.5 Instruct models, BitCal-TTS improves exact-match accuracy over a non-bit-aware adaptive baseline.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Post-training quantization makes large reasoning models practical under tight memory and latency budgets, but it can distort the online signals that drive adaptive test-time compute allocation. Under a fixed cap on the number of newly generated tokens, miscalibrated confidence can lead to harmful early halting: the model may surface a plausible final line while the underlying reasoning is still wrong, or the controller may stop before the trace has stabilized. We study this interaction for greedy 4-bit inference and propose BitCal-TTS, a lightweight runtime controller that combines (i) inexpensive online proxies for token-level uncertainty and reasoning-trace stability, (ii) a bit-conditioned confidence rescaling that is conservative at low nominal precision, and (iii) a bit-aware post-marker confirmation horizon designed for GSM8K-style structured outputs. The method requires no fine-tuning of the base model and integrates with standard Hugging Face 4-bit inference using forward hooks for logits and last-layer hidden states. On small evaluation shards of GSM8K with Qwen2.5 Instruct models, BitCal-TTS improves exact-match accuracy over a non-bit-aware adaptive baseline at the 7B and 14B scales while preserving substantial token savings relative to fixed-budget decoding. At a token cap of B=512, on the evaluation shards we report (N=54 for 7B and N=35 for 14B; not the full GSM8K test set), accuracy gains are +3.7 points (7B) and +2.8 points (14B), with the premature-stop rate falling from 14.8% to 11.1% on 7B and from 17.1% to 11.4% on 14B. We report Wilson 95% confidence intervals throughout and explicitly discuss the limited statistical power of the partial-shard comparisons. We release code and figure-generation scripts to support full reproduction.

Related papers

• State of Thought Enables Endogenous Reasoning [48.902117018115256]
  Test-time reasoning has emerged as a major approach to improving the capabilities of Large Language Models (LLMs)<n>We propose State of Thought, a new reasoning paradigm that enables endogenous reasoning in LLMs.<n>We show that SoT consistently improves mean-baseline accuracy while reducing generated tokens by 62.6% and end-to-end latency by 44.6%.
  arXiv  Detail & Related papers  (2026-09-13T05:55:03Z)
• Scaling Post-Training Ternarisation to Qwen3-8B Capability Retention, Reproduction, Lossless Packing, and Packed Execution [0.0]
  We study a scale-up of an aggressive post-training conversion pipeline from Qwen3-4B to Qwen3-8B.<n>The conversion uses KOTMS rotation, E2M-ATQ adaptive ternarisation, and GPTQ-style error compensation in a weight-only A16 configuration.
  arXiv  Detail & Related papers  (2026-09-08T06:21:44Z)
• Post-Training Ternarization of Qwen3-4B Capability, Effective Bit Budget, Storage Compression, and Deployment [0.0]
  Ultra-low-bit language models can reduce storage and memory bandwidth, but a nominal "1.58-bit" label does not fully describe the stored representation, retained capability, or runtime behavior.<n>We study an end-to-end post-training conversion of Qwen, an instruction-tuned 4B- parameter model, using KOTMS rotation, E2M-ATQ ternarization, and GPTQ-style error compensation from TWLA.<n>We evaluate effective bit accounting, task capability retention, perplexity, calibration sensitivity, checkpoint composition, and deployment behavior.
  arXiv  Detail & Related papers  (2026-09-02T00:42:54Z)
• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts [0.0]
  We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved.<n>We classify 30,000 chain-of-thought outputs from five instruction-tuned LLMs across three quantization precisions.<n>We find that while accuracy is robust across precisions, Hollow Convergence shows a significant size-dependent shift under NF4.
  arXiv  Detail & Related papers  (2026-07-10T21:55:05Z)
• Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning [18.69627681731888]
  Online Reasoning calibration (ORCA) is a framework for calibrating the sampling process that draws upon conformal prediction and test-time training.<n>At risk level $=0.1$, ORCA improves Qwen2.5-32B efficiency on in-distribution tasks with savings up to 47.5% with supervised labels and 40.7% with self-consistency labels.
  arXiv  Detail & Related papers  (2026-04-01T17:21:50Z)
• Uncertainty Makes It Stable: Curiosity-Driven Quantized Mixture-of-Experts [6.221156050218661]
  We present a curiosity-driven quantized Mixture-of-Experts framework for deep neural networks on resource-constrained devices.<n>Our 4-bit quantization maintains 99.9 percent of 16-bit accuracy (0.858 vs 0.859 F1) with 4x compression and 41 percent energy savings.<n>Our information-theoretic routing demonstrates that adaptive quantization yields accurate (0.858 F1, 1.2M params), energy-efficient (3.87 F1/mJ), and predictable edge models.
  arXiv  Detail & Related papers  (2025-11-13T15:32:41Z)
• Think Right: Learning to Mitigate Under-Over Thinking via Adaptive, Attentive Compression [68.69801176669843]
  We propose an online post-training RL method that prunes redundant steps and estimates difficulty.<n> TRAAC (Think Right with Adaptive, Attentive Compression) achieves an average absolute accuracy gain of 8.4%.<n>Although our models are trained on math datasets, they show accuracy and efficiency gains on out-of-distribution non-math datasets.
  arXiv  Detail & Related papers  (2025-10-02T02:00:20Z)
• Learnable Conformal Prediction with Context-Aware Nonconformity Functions for Robotic Planning and Perception [4.694504497452662]
  Learnable Conformal Prediction replaces fixed scores with a lightweight neural function to produce context-aware uncertainty sets.<n>It maintains CP's theoretical guarantees while reducing prediction set sizes by 18% in classification, tightening detection intervals by 52%, and improving path planning safety from 72% to 91% success with minimal overhead.<n> Hardware evaluation shows LCP adds less than 1% memory and 15.9% inference overhead, yet sustains 39 FPS on detection tasks while being 7.4 times more energy-efficient than ensembles.
  arXiv  Detail & Related papers  (2025-09-26T06:44:58Z)
• VeriThinker: Learning to Verify Makes Reasoning Model Efficient [52.74493506816969]
  Large Reasoning Models excel at complex tasks using Chain-of-Thought (CoT) reasoning.<n>Their tendency to overthinking leads to unnecessarily lengthy reasoning chains.<n>We introduce VeriThinker, a novel approach for CoT compression.
  arXiv  Detail & Related papers  (2025-05-23T14:17:56Z)
• Optimization of Quantum Error Correcting Code under Temporal Variation of Qubit Quality [2.348041867134616]
  We analyze 12 days of calibration data from IBM's 127-qubit device (ibm_kyiv)<n>We propose a simple adaptive QEC approach that selects an appropriate code distance per qubit, based on daily error rates.<n>Across 12 calibration days on ibm_kyiv, our adaptive strategy reduces physical qubit overhead by over 50% per logical qubit while maintaining access to 85-100% of usable qubits.
  arXiv  Detail & Related papers  (2025-05-09T16:15:17Z)
• Robust Conformal Prediction with a Single Binary Certificate [58.450154976190795]
  Conformal prediction (CP) converts any model's output to prediction sets with a guarantee to cover the true label with (adjustable) high probability.<n>We propose a robust conformal prediction that produces smaller sets even with significantly lower MC samples.
  arXiv  Detail & Related papers  (2025-03-07T08:41:53Z)
• Conservative Prediction via Data-Driven Confidence Minimization [70.93946578046003]
  In safety-critical applications of machine learning, it is often desirable for a model to be conservative. We propose the Data-Driven Confidence Minimization framework, which minimizes confidence on an uncertainty dataset.
  arXiv  Detail & Related papers  (2023-06-08T07:05:36Z)
• The case for 4-bit precision: k-bit Inference Scaling Laws [75.4335600212427]
  Quantization methods reduce the number of bits required to represent each parameter in a model. The final model size depends on both the number of parameters of the original model and the rate of compression. We run more than 35,000 zero-shot experiments with 16-bit inputs and k-bit parameters to examine which quantization methods improve scaling for 3 to 8-bit precision.
  arXiv  Detail & Related papers  (2022-12-19T18:48:33Z)
• HAWQV3: Dyadic Neural Network Quantization [73.11579145354801]
  Current low-precision quantization algorithms often have the hidden cost of conversion back and forth from floating point to quantized integer values. We present HAWQV3, a novel mixed-precision integer-only quantization framework.
  arXiv  Detail & Related papers  (2020-11-20T23:51:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.