PSViT: A Methodology for Structurally Pruning Spiking Vision Transformers

• URL: http://arxiv.org/abs/2606.03257v1
• Date: Tue, 02 Jun 2026 07:18:57 GMT
• Title: PSViT: A Methodology for Structurally Pruning Spiking Vision Transformers
• Abstract summary: Spiking Vision Transformer (SViT) models are promising low-power ViT models for solving vision-based tasks.<n>We propose PSViT, a novel methodology to perform structured pruning on SViT models.
• Score: 2.9583391652768314
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spiking Vision Transformer (SViT) models are promising low-power ViT models for solving vision-based tasks with state-of-the-art performance. However, their large sizes limit their deployments for resource-constrained embedded platforms, underscoring the needs of model compression. One of prominent compression techniques is pruning, and the state-of-the-art works employ unstructured pruning techniques to compress SViT models. Such techniques require specialized hardware architectures tailored for the sparsity patterns to maximize their efficiency benefits, making this approach not scalable. To address this, we propose PSViT, a novel methodology to perform structured pruning on SViT models, hence making it possible to efficiently accelerate their inference using the existing and widely-used computing architectures. To do this, PSViT employs several key steps: uniform channel-wise filter pruning to structurally eliminate the non-significant weights, sensitivity analysis to evaluate the impact of channel-wise pruning of individual layer on accuracy and network size, as well as fine-grained channel-wise pruning based on the sensitivity analysis and the given network architecture. Experimental results show that PSViT effectively obtains 22.4% memory saving through single-shot pruning, while maintaining high accuracy within 3% (70.3% without fine-tuning and 72.8% with fine-tuning) from the original non-pruned SViT model (73.3%) on the ImageNet-1K. These results also show that the PSViT methodology advances the effort in enabling efficient SViT deployments on resource-constrained applications.

Related papers

• Spatial-Aware Reduction Framework: Towards Efficient and Faithful Visual State Space Models [71.59431204970339]
  STORM is a spatial-aware token reduction framework.<n>It reformulates reduction into a structured operation on spatial units.<n>It achieves state-of-the-art pruning accuracy across diverse vision Mamba backbones.
  arXiv  Detail & Related papers  (2026-06-18T08:31:11Z)
• PrimeSVT: An Automated Memory-aware Pruning Framework with Prioritized Compression Policy for Spiking Vision Transformers [2.9583391652768314]
  Spiking Vision Transformers (SViTs) still hinder their embedded implementation, highlighting the need for model compression.<n>We propose PrimeSVT, a novel framework that performs automated memory-aware structured pruning on pre-trained SViT models.<n> Experimental results show that PrimeSVT saves 26.68% memory through automated single-shot pruning.
  arXiv  Detail & Related papers  (2026-06-02T10:18:00Z)
• QSViT: A Methodology for Quantizing Spiking Vision Transformers [5.343921650701002]
  Vision Transformer (ViT)-based models have shown state-of-the-art performance (e.g., accuracy) in vision-based AI tasks.<n>However, realizing their capability in resource-constrained embedded AI systems is challenging due to their inherent large memory footprints.<n>We propose QSViT, a novel design methodology to compress the SViT models through a systematic quantization strategy.
  arXiv  Detail & Related papers  (2025-04-01T16:34:46Z)
• ALoRE: Efficient Visual Adaptation via Aggregating Low Rank Experts [71.91042186338163]
  ALoRE is a novel PETL method that reuses the hypercomplex parameterized space constructed by Kronecker product to Aggregate Low Rank Experts.<n>Thanks to the artful design, ALoRE maintains negligible extra parameters and can be effortlessly merged into the frozen backbone.
  arXiv  Detail & Related papers  (2024-12-11T12:31:30Z)
• Comb, Prune, Distill: Towards Unified Pruning for Vision Model Compression [24.119415458653616]
  We propose a novel unified pruning framework Comb, Prune, Distill (CPD) to address both model-agnostic and task-agnostic concerns simultaneously. Our framework employs a combing step to resolve hierarchical layer-wise dependency issues, enabling architecture independence. In image classification we achieve a speedup of up to x4.3 with a accuracy loss of 1.8% and in semantic segmentation up to x1.89 with a 5.1% loss in mIoU.
  arXiv  Detail & Related papers  (2024-08-06T09:02:31Z)
• Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution [91.3781512926942]
  Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures. This work investigates the potential of network pruning for super-resolution iteration to take advantage of off-the-shelf network designs and reduce the underlying computational overhead. We propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly network at each and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly.
  arXiv  Detail & Related papers  (2023-03-16T21:06:13Z)
• GOHSP: A Unified Framework of Graph and Optimization-based Heterogeneous Structured Pruning for Vision Transformer [76.2625311630021]
  Vision transformers (ViTs) have shown very impressive empirical performance in various computer vision tasks. To mitigate this challenging problem, structured pruning is a promising solution to compress model size and enable practical efficiency. We propose GOHSP, a unified framework of Graph and Optimization-based Structured Pruning for ViT models.
  arXiv  Detail & Related papers  (2023-01-13T00:40:24Z)
• Efficient Neural Net Approaches in Metal Casting Defect Detection [0.0]
  This research proposes a lightweight architecture that is efficient in terms of accuracy and inference time. Our results indicate that a custom model of 590K parameters with depth-wise separable convolutions outperformed pretrained architectures.
  arXiv  Detail & Related papers  (2022-08-08T13:54:36Z)
• SmoothNets: Optimizing CNN architecture design for differentially private deep learning [69.10072367807095]
  DPSGD requires clipping and noising of per-sample gradients. This introduces a reduction in model utility compared to non-private training. We distilled a new model architecture termed SmoothNet, which is characterised by increased robustness to the challenges of DP-SGD training.
  arXiv  Detail & Related papers  (2022-05-09T07:51:54Z)
• Global Vision Transformer Pruning with Hessian-Aware Saliency [93.33895899995224]
  This work challenges the common design philosophy of the Vision Transformer (ViT) model with uniform dimension across all the stacked blocks in a model stage. We derive a novel Hessian-based structural pruning criteria comparable across all layers and structures, with latency-aware regularization for direct latency reduction. Performing iterative pruning on the DeiT-Base model leads to a new architecture family called NViT (Novel ViT), with a novel parameter that utilizes parameters more efficiently.
  arXiv  Detail & Related papers  (2021-10-10T18:04:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.