SSDi8: Accurate and Efficient 8-bit Quantization for State Space Duality

• URL: http://arxiv.org/abs/2608.21952v1
• Date: Sat, 22 Aug 2026 13:29:28 GMT
• Title: SSDi8: Accurate and Efficient 8-bit Quantization for State Space Duality
• Abstract summary: We present SSDi8, the first post-training quantization framework specifically designed for Structured State Space Duality (SSD)<n>SSDi8 adaptively quantizes channel-varying activations at cost-effective points, further reducing latency.<n>It achieves accuracy comparable to FP16 while delivering up to 1.4x speedup in W4A8 and W8A8 settings.
• Score: 49.80188257904718
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in sequence modeling have highlighted Mamba as a state space architecture offering efficient long-range dependency modeling and providing a viable alternative to Transformers. Building upon this, Mamba-2 introduces the Structured State Space Duality (SSD), which integrates recurrent and attention modes to achieve efficiency and scalability. However, this architectural expansion substantially increases memory and latency overhead, underscoring the need for efficient compression strategies tailored to SSD. In this work, we present SSDi8, the first post-training quantization framework specifically designed for SSD to maintain a persistent INT8 path. SSDi8 introduces a reformulation that decouples element-wise multiplications from matrix multiplications, enabling reuse of quantized activations across modules. Moreover, SSDi8 adaptively quantizes channel-varying activations at cost-effective points, further reducing latency. On the accuracy side, SSDi8 explicitly leverages the intrinsic dimensional decomposition of SSD, exploiting distinct outlier distributions across axes, and incorporates an error correction term based on per-channel error statistics. Comprehensive experiments demonstrate that SSDi8 achieves accuracy comparable to FP16 while delivering up to 1.4x speedup in W4A8 and W8A8 settings. We further validate its robustness in resource-constrained environments by deploying it on the Orin NX device.

Related papers

• QuantSR+: Pushing the Limit of Quantized Image Super-Resolution Networks [63.04549067734431]
  Low-bit quantization is widely used to compress super-resolution (SR) models.<n>However, when SR models are pushed to ultra-low precision (2-4 bits), performance can drop sharply.<n>We propose QuantSR+, a unified framework that improves quantization operators, network design, and training optimization.
  arXiv  Detail & Related papers  (2026-05-21T11:38:52Z)
• OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond [50.440302567029654]
  Multi-modal intelligence has made the Key-Value cache a dominant memory bottleneck for efficient deployment.<n>In this work, we revisit the inherent limitations of the per-channel quantization paradigm.<n>We propose OScaR, an accurate and lightweight KV cache compression framework for X-LLMs.
  arXiv  Detail & Related papers  (2026-05-19T10:53:03Z)
• BabyMamba-HAR: Lightweight Selective State Space Models for Efficient Human Activity Recognition on Resource Constrained Devices [0.0]
  Human activity recognition (HAR) on wearable and mobile devices is constrained by memory footprint and computational budget.<n> Selective state space models (SSMs) offer linear time sequence processing with input dependent gating.<n>BabyMamba-HAR is introduced, a framework comprising two novel lightweight Mamba inspired architectures for resource constrained HAR.
  arXiv  Detail & Related papers  (2026-02-10T15:16:32Z)
• The Curious Case of In-Training Compression of State Space Models [49.819321766705514]
  State Space Models (SSMs) tackle long sequence modeling tasks efficiently, offer both parallelizable training and fast inference.<n>Key design challenge is striking the right balance between maximizing expressivity and limiting this computational burden.<n>Our approach, textscCompreSSM, applies to Linear Time-Invariant SSMs such as Linear Recurrent Units, but is also extendable to selective models.
  arXiv  Detail & Related papers  (2025-10-03T09:02:33Z)
• Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models [23.045441347570886]
  State Space Models (SSMs) are emerging as a compelling alternative to Transformers because of their consistent memory usage and high performance.<n>To overcome this, quantizing SSMs with low bit-width data formats can reduce model size and benefit from hardware acceleration.<n>We present Quamba2, compatible with W8A8, W4A8, and W4A16 for both Mamba1 and Mamba2 backbones.
  arXiv  Detail & Related papers  (2025-03-28T21:10:39Z)
• Quamba: A Post-Training Quantization Recipe for Selective State Space Models [8.924779222965798]
  State Space Models (SSMs) have emerged as an appealing alternative to Transformers for large language models.<n>We propose a static 8-bit per-tensor SSM quantization method which suppresses the maximum values of the input activations to the selective SSM.<n>Our 8-bit weight-activation quantized Mamba 2.8B SSM benefits from hardware acceleration and achieves a 1.72x lower generation latency on an Nvidia Orin Nano 8G, with only a 0.9% drop in average accuracy on zero-shot tasks.
  arXiv  Detail & Related papers  (2024-10-17T05:32:33Z)
• DeepGEMM: Accelerated Ultra Low-Precision Inference on CPU Architectures using Lookup Tables [49.965024476651706]
  DeepGEMM is a lookup table based approach for the execution of ultra low-precision convolutional neural networks on SIMD hardware. Our implementation outperforms corresponding 8-bit integer kernels by up to 1.74x on x86 platforms.
  arXiv  Detail & Related papers  (2023-04-18T15:13:10Z)
• 8-bit Optimizers via Block-wise Quantization [57.25800395197516]
  Statefuls maintain statistics over time, e.g., the exponentially smoothed sum (SGD with momentum) or squared sum (Adam) of past values. This state can be used to accelerate optimization compared to plain gradient descent but uses memory that might otherwise be allocated to model parameters. In this paper, we develop first gradients that use 8-bit statistics while maintaining the performance levels of using 32-bit gradient states.
  arXiv  Detail & Related papers  (2021-10-06T15:43:20Z)
• Systolic Tensor Array: An Efficient Structured-Sparse GEMM Accelerator for Mobile CNN Inference [16.812184391068786]
  Convolutional neural network (CNN) inference on mobile devices demands efficient hardware acceleration. systolic array (SA) is a pipelined 2D array of processing elements (PEs) We describe two significant improvements to the traditional SA architecture, to specifically optimize for CNN inference.
  arXiv  Detail & Related papers  (2020-05-16T20:47:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.