S$^2$Prune: Spatially Structured Visual Token Pruning for Multimodal Large Language Models

• URL: http://arxiv.org/abs/2609.01224v1
• Date: Tue, 01 Sep 2026 13:26:57 GMT
• Title: S$^2$Prune: Spatially Structured Visual Token Pruning for Multimodal Large Language Models
• Abstract summary: Visual token pruning reduces the inference overhead of multimodal large language models (MLLMs) by retaining only a subset of visual tokens.<n>We propose S$2$Prune, a training-free pruning method that preserves spatial coverage while adapting token density to local image structure.
• Score: 5.680072178589744
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual token pruning reduces the inference overhead of multimodal large language models (MLLMs) by retaining only a subset of visual tokens. Existing methods usually select tokens based on importance or redundancy. However, we observe that these criteria produce stable spatial biases across inputs and do not always outperform simple Uniform Grid sampling, highlighting the value of broad spatial coverage. Motivated by this, we propose S$^2$Prune, a training-free pruning method that preserves spatial coverage while adapting token density to local image structure. We first divide the image into regions and assign at least one token to each region to preserve coverage. The remaining token budget is then distributed according to Laplacian variation, giving more tokens to regions with richer structure. We then use Early Representation Change (ERC), computed from the first decoder block, to select representative tokens within each region. We evaluate S$^2$Prune across diverse settings and two MLLM architectures. On Qwen2.5-VL-7B-Instruct, it achieves the highest average accuracy among the evaluated training-free pruning methods. With only 32 of the original 576 visual tokens, it still retains 79.3% of the full-model performance. Code is available at https://github.com/yuanyuanjia71-spec/S2Prune.

Related papers

• CoVeR: Coverage-Based Token Pruning for Multi-View 3D Reasoning in VLMs [58.22241532542095]
  Representing a 3D scene as multi-view images allows 2D VLMs to reason in 3D by reusing priors from pre-training, sidestepping the scarcity of annotated 3D data.<n>Existing visual token pruners fall into two families, each limited in the 3D multi-view setting.<n>We show that spatial coverage is associated with 3D reasoning performance and introduce CoVeR, a deterministic, training-free selector that uses only token coordinates, with no learned signals.
  arXiv  Detail & Related papers  (2026-09-08T07:16:37Z)
• HAP: Head-Adaptive Visual Token Pruning via Cross-Modal Alignment [76.68875424389057]
  Existing methods score each visual token by averaging text-to-visual attention uniformly across all heads.<n> misaligned heads dominate the average, amplifying background tokens and drowning out fine-grained cues.<n>We propose PAQ (Prompt-Grounded Attention Quality), a metric quantifying how well each head aligns the prompt with image regions.
  arXiv  Detail & Related papers  (2026-08-24T23:55:36Z)
• CoverPrune: Coverage-Driven Token Pruning for 3D VLMs via Optimal Transport [50.2610873651284]
  Existing token pruning methods rely on diversity-based selection, discarding similar tokens to maximize dispersion.<n>In 3D environments, this approach frequently drops representative prototype tokens in favor of outliers.<n>We propose a paradigm shift for 3D VLM token pruning: from maximizing OT to preserving visual evidence coverage.
  arXiv  Detail & Related papers  (2026-08-13T13:29:15Z)
• How Many Visual Tokens Do Multimodal Language Models Need? Scaling Visual Token Pruning with F^3A [35.028071915491104]
  We propose F3A, a training-free router for visual token pruning.<n>It builds lightweight question-conditioned cues, matches them to visual-grid tokens through frozen sparse sensing heads.<n>It allocates a fixed vision token budget via coarse evidence localization, local refinement, coverage-preserving competition, and recovery of under-covered regions.
  arXiv  Detail & Related papers  (2026-05-09T13:13:04Z)
• ReDiPrune: Relevance-Diversity Pre-Projection Token Pruning for Efficient Multimodal LLMs [16.523460406504604]
  ReDiPrune is a training-free token pruning method applied before the vision-conditioned projector.<n>It selects informative tokens directly from vision encoder outputs, preserving fine-grained spatial and semantic cues.<n>It consistently improves the accuracy-efficiency trade-off across four video and five image benchmarks.
  arXiv  Detail & Related papers  (2026-03-25T18:01:19Z)
• IVC-Prune: Revealing the Implicit Visual Coordinates in LVLMs for Vision Token Pruning [27.75049214892312]
  Large Vision-Language Models (LVLMs) achieve impressive performance across multiple tasks.<n>A significant challenge, however, is their prohibitive inference cost when processing high-resolution visual inputs.<n>We propose textbfIVC-Prune, a training-free, prompt-aware pruning strategy that retains both IVC tokens and semantically relevant foreground tokens.
  arXiv  Detail & Related papers  (2026-02-03T03:39:31Z)
• GridPrune: From "Where to Look" to "What to Select" in Visual Token Pruning for MLLMs [2.9869094956508495]
  Multimodal large language models (MLLMs) have shown remarkable capabilities in a wide range of vision-language tasks.<n>Visual token pruning has emerged as a key technique for enhancing the efficiency of MLLMs.<n>We propose GridPrune, a method that replaces the global Top-K mechanism with a "guide-globally, select-locally" zonal selection system.
  arXiv  Detail & Related papers  (2025-11-13T08:35:39Z)
• SCOPE: Saliency-Coverage Oriented Token Pruning for Efficient Multimodel LLMs [59.415473779171315]
  We propose a novel visual token pruning strategy called textbfSaliency-textbfCoverage textbfOriented token textbfPruning for textbfEfficient MLLMs.
  arXiv  Detail & Related papers  (2025-10-28T09:29:37Z)
• Tokenize Anything via Prompting [65.93061853439512]
  We present a unified, promptable model capable of simultaneously segmenting, recognizing, and captioning anything. We train a generalizable model with massive segmentation masks, eg, SA-1B masks, and semantic priors from a pre-trained CLIP model with 5 billion parameters. We believe this model can be a versatile region-level image tokenizer, capable of encoding general-purpose region context.
  arXiv  Detail & Related papers  (2023-12-14T17:01:02Z)
• Improving Robustness and Generality of NLP Models Using Disentangled Representations [62.08794500431367]
  Supervised neural networks first map an input $x$ to a single representation $z$, and then map $z$ to the output label $y$. We present methods to improve robustness and generality of NLP models from the standpoint of disentangled representation learning. We show that models trained with the proposed criteria provide better robustness and domain adaptation ability in a wide range of supervised learning tasks.
  arXiv  Detail & Related papers  (2020-09-21T02:48:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.