Ternary Mamba: Grouped Quantization-Aware Training of W1.58A16 State Space Models

• URL: http://arxiv.org/abs/2606.18114v1
• Date: Tue, 16 Jun 2026 16:18:21 GMT
• Title: Ternary Mamba: Grouped Quantization-Aware Training of W1.58A16 State Space Models
• Abstract summary: State Space Models (SSMs) such as Mamba-2 offer linear-time inference but their memory footprint limits edge deployment.<n>We show a pretrained checkpoint suffices, reducing the marginal token budget by 1,000x.<n>We achieve 48.1% zero-shot accuracy in just 102M tokens (4 GPU-hours, single H100)
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: State Space Models (SSMs) such as Mamba-2 offer linear-time inference but their memory footprint limits edge deployment. Prior ternary SSM work (Slender-Mamba) trains from scratch on 150B tokens; we show a pretrained checkpoint suffices, reducing the marginal token budget by 1,000x. Using grouped quantization-aware training (QAT) with knowledge distillation from a frozen FP16 teacher, we compress Mamba-2 1.3B to 3.61x (2,687 to 744 MB) and achieve 48.1% zero-shot accuracy (7-task average) in just 102M tokens (4 GPU-hours, single H100) -- approaching Bi-Mamba's 48.4% (within +/-0.9pp CI). This QAT-from-pretrained setting reveals zero-ratio collapse, a novel instability caused by learnable quantization scales that does not arise in from-scratch training. We further show that post-hoc correction strategies effective for Transformers fail for SSMs due to error accumulation through the recurrence. These results demonstrate that ternary SSMs do not require expensive from-scratch training: QAT from pretrained checkpoints with KD is a data-efficient alternative.

Related papers

• Post-Training Ternarization of Qwen3-4B Capability, Effective Bit Budget, Storage Compression, and Deployment [0.0]
  Ultra-low-bit language models can reduce storage and memory bandwidth, but a nominal "1.58-bit" label does not fully describe the stored representation, retained capability, or runtime behavior.<n>We study an end-to-end post-training conversion of Qwen, an instruction-tuned 4B- parameter model, using KOTMS rotation, E2M-ATQ ternarization, and GPTQ-style error compensation from TWLA.<n>We evaluate effective bit accounting, task capability retention, perplexity, calibration sensitivity, checkpoint composition, and deployment behavior.
  arXiv  Detail & Related papers  (2026-09-02T00:42:54Z)
• Max-Window Scale Estimation for Near-Lossless HiF8 W8A8 Quantization-Aware Training [5.976336341654394]
  Quantization-aware training (QAT) with low-bit floating-point formats enables efficient LLM deployment, yet introduces subtle failure modes invisible to standard training metrics.<n>We present a systematic study of HiF8 W8A8 QAT for OpenPangu-Embedded-1B through the lens of Delayed Scaling (DTS)<n>We identify and disentangle two failure modes: (i)amax saturation, where delayed scale estimates corrupt knowledge-sensitive representations via forward-pass clipping, and (ii)catastrophic forgetting, where an aggressive learning rate overwrites pretrained commonsense knowledge independently of quantization
  arXiv  Detail & Related papers  (2026-05-25T09:19:57Z)
• ZAYA1-8B Technical Report [14.894881882111605]
  We present ZAYA1-8B, a mixture-of-experts (MoE) model with 700M active and 8B total parameters.<n>ZAYA1-8B matches or exceeds DeepSeek-R1-0528 on several challenging mathematics and coding benchmarks.<n>Post-training uses a four-stage RL cascade: reasoning warmup on math and puzzles; a 400-task RLVE-Gym curriculum; math and code RL with test-time compute traces and synthetic code environments.
  arXiv  Detail & Related papers  (2026-05-06T18:44:08Z)
• Density Field State Space Models: 1-Bit Distillation, Efficient Inference, and Knowledge Organization in Mamba-2 [0.0]
  We present Density Field State Space Models (DF-SSM), a framework for compressing SSMs to a 1-bit scaffold with int8 low-rank correction.<n>We achieve a 278 MB model (9.7x smaller than the 2.7 GB FP16 teacher) that runs at 21.4x faster inference on GPU.
  arXiv  Detail & Related papers  (2026-04-28T14:25:10Z)
• KnowRL: Boosting LLM Reasoning via Reinforcement Learning with Minimal-Sufficient Knowledge Guidance [50.70511573232489]
  We propose textbfKnowRL (Knowledge-Guided Reinforcement Learning), an RL training framework that treats hint design as a minimal-sufficient guidance problem.<n>KnowRL decomposes guidance into atomic knowledge points (KPs) and uses Constrained Subset Search (CSS) to construct compact, interaction-aware subsets for training.<n>Across eight reasoning benchmarks at the 1.5B scale, KnowRL-Nemotron-1.5B consistently outperforms strong RL and hinting baselines.
  arXiv  Detail & Related papers  (2026-04-14T11:53:23Z)
• StateSMix: Online Lossless Compression via Mamba State Space Models and Sparse N-gram Context Mixing [0.0]
  StateSMix is an online-trained Mamba-style State Space Model (SSM) with sparse n-gram context mixing and arithmetic coding.<n>The SSM provides a continuously-updated probability estimate over BPE tokens, while nine sparse n-gram hash tables add exact local and long-range pattern memorisation.<n>On the standard enwik8 benchmark, StateSMix achieves 2.123 bpb on 1 MB, 2.149 bpb on 3 MB, and 2.162 bpb on 10 MB, beating xz -9e (LZMA2) by 8.7%, 5.4%, and
  arXiv  Detail & Related papers  (2026-04-05T19:28:11Z)
• Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs [80.72350166388601]
  Nemotron Elastic is a framework for building reasoning-oriented LLMs.<n>It embeds nested submodels within a single parent model.<n>Each of these submodels shares weights with the parent model and can be extracted zero-shot during deployment.
  arXiv  Detail & Related papers  (2025-11-20T18:59:21Z)
• Mamba-OTR: a Mamba-based Solution for Online Take and Release Detection from Untrimmed Egocentric Video [57.805927523341516]
  Mamba-OTR is designed to exploit temporal recurrence during inference while being trained on short video clips.<n>Mamba-OTR achieves a noteworthy mp-mAP of 45.48 when operating in a sliding-window fashion.<n>We will publicly release the source code of Mamba-OTR to support future research.
  arXiv  Detail & Related papers  (2025-07-22T08:23:51Z)
• Mixture-of-Mamba: Enhancing Multi-Modal State-Space Models with Modality-Aware Sparsity [56.0251572416922]
  State Space Models (SSMs) have emerged as efficient alternatives to Transformers for sequential modeling.<n>We propose a novel SSM architecture that introduces modality-aware sparsity through modality-specific parameterization of the Mamba block.<n>We evaluate Mixture-of-Mamba across three multi-modal pretraining settings.
  arXiv  Detail & Related papers  (2025-01-27T18:35:05Z)
• The Mamba in the Llama: Distilling and Accelerating Hybrid Models [76.64055251296548]
  We show how to distill large Transformers into linear RNNs by reusing the linear projection weights from attention layers with academic GPU resources.<n>The resulting hybrid model achieves performance comparable to the original Transformer in chat benchmarks.<n>We also introduce a hardware-aware speculative decoding algorithm that accelerates the inference speed of Mamba and hybrid models.
  arXiv  Detail & Related papers  (2024-08-27T17:56:11Z)
• An Empirical Study of Mamba-based Language Models [69.74383762508805]
  Selective state-space models (SSMs) like Mamba overcome some shortcomings of Transformers. We present a direct comparison between 8B-context Mamba, Mamba-2, and Transformer models trained on the same datasets. We find that the 8B Mamba-2-Hybrid exceeds the 8B Transformer on all 12 standard tasks.
  arXiv  Detail & Related papers  (2024-06-12T05:25:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.