QuarterMap: Efficient Post-Training Token Pruning for Visual State Space Models

• URL: http://arxiv.org/abs/2507.09514v1
• Date: Sun, 13 Jul 2025 06:49:32 GMT
• Title: QuarterMap: Efficient Post-Training Token Pruning for Visual State Space Models
• Abstract summary: QuarterMap removes redundant spatial activations before scanning and restores dimensions via nearest-neighbor upsampling.<n>On ImageNet-1K, QuarterMap up to 11% speedup on VMamba with less than 0.9% accuracy drop, and yields similar gains on ADE20K segmentation.<n>We validate QuarterMap on MedMamba, a domain-specific model that shares the same four-directional scanning structure.
• Score: 8.543701468361395
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: State space models (SSMs) reduce the quadratic complexity of transformers by leveraging linear recurrence. Recently, VMamba has emerged as a strong SSM-based vision backbone, yet remains bottlenecked by spatial redundancy in its four-directional scan. We propose QuarterMap, a post-training activation pruning method that removes redundant spatial activations before scanning and restores dimensions via nearest-neighbor upsampling. Our method improves throughput without retraining. On ImageNet-1K, QuarterMap achieves up to 11% speedup on VMamba with less than 0.9% accuracy drop, and yields similar gains on ADE20K segmentation. Beyond VMamba, we validate QuarterMap on MedMamba, a domain-specific model that shares the same four-directional scanning structure, where it consistently improves throughput while preserving accuracy across multiple medical imaging tasks. Compared to token merging methods like ToMe, QuarterMap is tailored for SSMs and avoids costly merge-unmerge operations. Our method offers a plug-and-play tool for deployment-time efficiency without compromising transferability.

Related papers

• Spatial-Aware Reduction Framework: Towards Efficient and Faithful Visual State Space Models [71.59431204970339]
  STORM is a spatial-aware token reduction framework.<n>It reformulates reduction into a structured operation on spatial units.<n>It achieves state-of-the-art pruning accuracy across diverse vision Mamba backbones.
  arXiv  Detail & Related papers  (2026-06-18T08:31:11Z)
• No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval [51.43543998583709]
  Single-stage Sparse Retrieval (SSR) is a paradigm shift that replaces expensive clustering with efficient sparse coding.<n>It reduces indexing time by 15x compared to ColBERTv2, halves retrieval latency, and simultaneously improves retrieval performance.
  arXiv  Detail & Related papers  (2026-05-28T15:53:34Z)
• Split4D: Decomposed 4D Scene Reconstruction Without Video Segmentation [76.21162972133534]
  We represent a decomposed 4D scene with Freetime FeatureGS.<n>We design a streaming feature learning strategy to accurately recover it from per-image segmentation maps.<n> Experimental results on several datasets show that the reconstruction quality of our method outperforms recent methods by a large margin.
  arXiv  Detail & Related papers  (2025-12-28T02:37:12Z)
• Sequential Token Merging: Revisiting Hidden States [29.89737086314315]
  Vision Mambas (ViMs) achieve remarkable success with sub-quadratic complexity, but their efficiency remains constrained by quadratic token scaling with image resolution.<n>We propose Sequential Token Merging (STM), featuring: 1) Bidirectional nearest neighbor merging to preserve sequential dependencies through symmetric spatial aggregation, and 2) Hidden states protection to stabilize the hidden states around the class token.<n>Our method achieves state-of-the-art efficiency with minimal complexity, while providing new insights into state-space model dynamics.
  arXiv  Detail & Related papers  (2025-09-19T14:45:01Z)
• MambaMap: Online Vectorized HD Map Construction using State Space Model [11.15033113060733]
  MambaMap is a novel framework that efficiently fuses long-range temporal features in the state space to construct online vectorized HD maps.<n>Specifically, MambaMap incorporates a memory bank to store and utilize information from historical frames.<n>In addition, we design innovative multi-directional and spatial-temporal scanning strategies to enhance feature extraction at both BEV and instance levels.
  arXiv  Detail & Related papers  (2025-07-27T11:09:27Z)
• StruMamba3D: Exploring Structural Mamba for Self-supervised Point Cloud Representation Learning [31.585380521480868]
  We propose StruMamba3D, a novel paradigm for self-supervised point cloud representation learning.<n>We design spatial states and use them as proxies to preserve spatial dependencies among points.<n>Our method attains the SOTA 95.1% accuracy on ModelNet40 and 92.75% accuracy on the most challenging split of ScanObjectNN without voting strategy.
  arXiv  Detail & Related papers  (2025-06-26T17:58:05Z)
• V"Mean"ba: Visual State Space Models only need 1 hidden dimension [0.7864304771129751]
  State Space Models (SSMs) have emerged as a solution by introducing a linear recurrence mechanism.<n>We introduce textitVMeanba, a training-free compression method that eliminates the channel dimension in SSMs using mean operations.<n>We show that textitVMeanba achieves up to a 1.12x speedup with less than a 3% accuracy loss.
  arXiv  Detail & Related papers  (2024-12-21T12:27:07Z)
• MambaIRv2: Attentive State Space Restoration [96.4452232356586]
  Mamba-based image restoration backbones have recently demonstrated significant potential in balancing global reception and computational efficiency.<n>We propose MambaIRv2, which equips Mamba with the non-causal modeling ability similar to ViTs to reach the attentive state space restoration model.
  arXiv  Detail & Related papers  (2024-11-22T12:45:12Z)
• MambaVC: Learned Visual Compression with Selective State Spaces [74.29217829932895]
  We introduce MambaVC, a simple, strong and efficient compression network based on SSM. MambaVC develops a visual state space (VSS) block with a 2D selective scanning (2DSS) module as the nonlinear activation function after each downsampling. On compression benchmark datasets, MambaVC achieves superior rate-distortion performance with lower computational and memory overheads.
  arXiv  Detail & Related papers  (2024-05-24T10:24:30Z)
• Coordinate Transformer: Achieving Single-stage Multi-person Mesh Recovery from Videos [91.44553585470688]
  Multi-person 3D mesh recovery from videos is a critical first step towards automatic perception of group behavior in virtual reality, physical therapy and beyond. We propose the Coordinate transFormer (CoordFormer) that directly models multi-person spatial-temporal relations and simultaneously performs multi-mesh recovery in an end-to-end manner. Experiments on the 3DPW dataset demonstrate that CoordFormer significantly improves the state-of-the-art, outperforming the previously best results by 4.2%, 8.8% and 4.7% according to the MPJPE, PAMPJPE, and PVE metrics, respectively.
  arXiv  Detail & Related papers  (2023-08-20T18:23:07Z)
• NeuMap: Neural Coordinate Mapping by Auto-Transdecoder for Camera Localization [60.73541222862195]
  NeuMap is an end-to-end neural mapping method for camera localization. It encodes a whole scene into a grid of latent codes, with which a Transformer-based auto-decoder regresses 3D coordinates of query pixels.
  arXiv  Detail & Related papers  (2022-11-21T04:46:22Z)
• CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point Cloud Learning [81.85951026033787]
  We set transformers in this work and incorporate them into a hierarchical framework for shape classification and part and scene segmentation. We also compute efficient and dynamic global cross attentions by leveraging sampling and grouping at each iteration. The proposed hierarchical model achieves state-of-the-art shape classification in mean accuracy and yields results on par with the previous segmentation methods.
  arXiv  Detail & Related papers  (2022-07-31T21:39:15Z)
• Trans4Map: Revisiting Holistic Top-down Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers [34.6312362205904]
  We propose an end-to-end one-stage Transformer-based framework for Mapping, termed Trans4Map. Trans4Map achieves state-of-the-art results, reducing 67.2% parameters, yet gaining a +3.25% mIoU and a +4.09% mBF1 improvements on the Matterport3D dataset.
  arXiv  Detail & Related papers  (2022-07-13T14:01:00Z)
• Generic Merging of Structure from Motion Maps with a Low Memory Footprint [3.7838598767969502]
  We present new tools that will enable efficient, flexible and robust map merging. Using both simulated and real data - from both a hand held mobile phone and from a drone - we verify the performance of the proposed method.
  arXiv  Detail & Related papers  (2021-03-24T15:03:25Z)
• HM4: Hidden Markov Model with Memory Management for Visual Place Recognition [54.051025148533554]
  We develop a Hidden Markov Model approach for visual place recognition in autonomous driving. Our algorithm, dubbed HM$4$, exploits temporal look-ahead to transfer promising candidate images between passive storage and active memory. We show that this allows constant time and space inference for a fixed coverage area.
  arXiv  Detail & Related papers  (2020-11-01T08:49:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.