Spatial-Aware Reduction Framework: Towards Efficient and Faithful Visual State Space Models

• URL: http://arxiv.org/abs/2606.19932v1
• Date: Thu, 18 Jun 2026 08:31:11 GMT
• Title: Spatial-Aware Reduction Framework: Towards Efficient and Faithful Visual State Space Models
• Abstract summary: STORM is a spatial-aware token reduction framework.<n>It reformulates reduction into a structured operation on spatial units.<n>It achieves state-of-the-art pruning accuracy across diverse vision Mamba backbones.
• Score: 71.59431204970339
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mamba demonstrates strong efficiency in modeling long visual sequences. However, when token reduction is applied to structurally enhanced Mamba variants, these models exhibit a severe performance collapse. We attribute this degradation to the spatially agnostic nature of existing reduction methods, which violate the two-dimensional structural premise required by the selective scanning mechanism. In this work, we propose STORM, a spatial-aware token reduction framework designed to maintain structural integrity throughout the compression process. STORM reformulates reduction into a structured operation on spatial units, enforcing localized constraints to maintain both grid topology and neighborhood coherence. As a plug-and-play module, STORM equips existing reduction pipelines with explicit spatial awareness without any training. Empirical results demonstrate that STORM achieves state-of-the-art pruning accuracy across diverse vision Mamba backbones under training-free settings. Notably, STORM delivers a substantial accuracy recovery on VMamba, outperforming prior methods by up to 63.3\% in top-1 accuracy. Meanwhile, STORM incurs only a 1.0\% accuracy drop on PlainMamba, achieving performance comparable to ViT.

Related papers

• Local Epistemic Uncertainty Guided Active Sampling for Plug-and-play Diffusive Image Restoration [15.371827475537776]
  Local Epistemic Uncertainty Guided Active Sampling framework is developed.<n>It balances detail preservation and artifact suppression in the spatial domain.<n>It reduces sampling time with negligible memory overhead.
  arXiv  Detail & Related papers  (2026-08-07T09:00:20Z)
• MambaADv2: Evolving Duality-enhanced State Space Model for Unsupervised Anomaly Detection [108.36437360254605]
  This paper proposes MambaADv2, a framework tailored for multi-class unsupervised anomaly detection.<n>MambaADv2 comprises a pre-trained encoder and a Mamba-inspired decoder, equipped with Duality-enhanced State Space (DSS) modules across multiple scales.<n>The structure of the Hybrid State Space (HSS) block is tailored by following the SSD-based Mamba lineage and incorporating Mamba3-style position-aware state-space modeling.
  arXiv  Detail & Related papers  (2026-06-22T10:14:06Z)
• DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Tasks [54.32016216994156]
  We introduce the Causal Latent World Model (CLWM), which employs DINOv3 features as generative targets to disentangle interaction semantics from visual noise.<n>To overcome memory scaling, CLWM features a Dual-State Test-Time Training (TTT) Memory that guarantees a strict $mathcalO(1)$ footprint for long-horizon tasks.<n>We present EmbodiChain, an online framework that establishes the Efficiency Law by injecting an infinite flow of physics-grounded trajectories during training.
  arXiv  Detail & Related papers  (2026-04-13T03:19:36Z)
• DLRMamba: Distilling Low-Rank Mamba for Edge Multispectral Fusion Object Detection [7.883488411699879]
  Multispectral fusion object detection is a critical task for edge-based maritime surveillance and remote sensing.<n>Current State Space Models (SSMs) like Mamba suffer from significant parameter redundancy in their standard 2D Selective Scan (SS2D) blocks.<n>We propose the Low-Rank Two-Dimensional Selective Structured State Space Model (Low-Rank SS2D), which reformulates state transitions via matrix factorization to exploit intrinsic feature sparsity.
  arXiv  Detail & Related papers  (2026-03-06T22:43:57Z)
• Interpreting and Steering State-Space Models via Activation Subspace Bottlenecks [6.542236618174232]
  State-space models (SSMs) have emerged as an efficient strategy for building powerful language models.<n>Despite their promise, the interpretability and steerability of modern SSMs remain relatively underexplored.<n>We identify activation subspace bottlenecks in the Mamba family of SSM models using tools from mechanistic interpretability.
  arXiv  Detail & Related papers  (2026-02-26T07:46:42Z)
• The Curious Case of In-Training Compression of State Space Models [49.819321766705514]
  State Space Models (SSMs) tackle long sequence modeling tasks efficiently, offer both parallelizable training and fast inference.<n>Key design challenge is striking the right balance between maximizing expressivity and limiting this computational burden.<n>Our approach, textscCompreSSM, applies to Linear Time-Invariant SSMs such as Linear Recurrent Units, but is also extendable to selective models.
  arXiv  Detail & Related papers  (2025-10-03T09:02:33Z)
• SparseSSM: Efficient Selective Structured State Space Models Can Be Pruned in One-Shot [8.080568103779893]
  State-space language models such as Mamba match Transformer quality while permitting linear complexity inference.<n>Existing one-shot pruning methods are tailored to attention blocks and fail to account for the time-shared and discretized state-transition matrix.<n>We introduce SparseSSM, the first training-free pruning framework that extends the classic optimal brain surgeon (OBS) framework to state space architectures.
  arXiv  Detail & Related papers  (2025-06-11T11:14:57Z)
• Efficient Unstructured Pruning of Mamba State-Space Models for Resource-Constrained Environments [6.908972852063454]
  State-space models (SSMs) have emerged as powerful alternatives to Transformers for sequence modeling.<n>We propose a novel unstructured pruning framework tailored for Mamba models that achieves up to 70% parameter reduction while retaining over 95% of the original performance.
  arXiv  Detail & Related papers  (2025-05-13T07:23:08Z)
• ACMamba: Fast Unsupervised Anomaly Detection via An Asymmetrical Consensus State Space Model [51.83639270669481]
  Unsupervised anomaly detection in hyperspectral images (HSI) aims to detect unknown targets from backgrounds.<n>HSI studies are hindered by steep computational costs due to the high-dimensional property of HSI and dense sampling-based training paradigm.<n>We propose an Asymmetrical Consensus State Space Model (ACMamba) to significantly reduce computational costs without compromising accuracy.
  arXiv  Detail & Related papers  (2025-04-16T05:33:42Z)
• Mamba-CL: Optimizing Selective State Space Model in Null Space for Continual Learning [54.19222454702032]
  Continual Learning aims to equip AI models with the ability to learn a sequence of tasks over time, without forgetting previously learned knowledge.<n>State Space Models (SSMs) have achieved notable success in computer vision.<n>We introduce Mamba-CL, a framework that continuously fine-tunes the core SSMs of the large-scale Mamba foundation model.
  arXiv  Detail & Related papers  (2024-11-23T06:36:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.