AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning

• URL: http://arxiv.org/abs/2607.07033v3
• Date: Wed, 15 Jul 2026 09:13:20 GMT
• Title: AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning
• Abstract summary: Large vision-language models incur substantial inference costs because high-resolution inputs introduce thousands of visual tokens.<n>We introduce AnchorPrune, a training-free framework that first constructs a protected relevance anchor and then expands it with visual context.<n>On LLaVA-NeXT-7B, AnchorPrune preserves 97.6% of full-token performance using only 160 of 2,880 visual tokens.
• Score: 8.862195491555575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large vision-language models incur substantial inference costs because high-resolution inputs introduce thousands of visual tokens, many of which are redundant for a given query. Existing pruning methods often combine query relevance and token diversity, yet these objectives can conflict under aggressive compression: relevance-driven selection may overconcentrate the budget on correlated local evidence, while diversity-driven selection may suppress indispensable tokens or retain distinct but uninformative regions. We introduce AnchorPrune, a training-free framework that first constructs a protected relevance anchor and then expands it with complementary visual context. AnchorPrune adaptively determines the anchor size from the novelty profile of relevance-ranked tokens, preserving a compact set of query-critical evidence, and allocates the remaining budget through importance-weighted novelty to recover informative, non-redundant context relative to the anchor. This ordered design prevents contextual expansion from displacing indispensable query cues while improving overall visual coverage. AnchorPrune is lightweight, architecture-aware, and requires neither retraining nor model modification. Across image and video vision-language models and benchmarks, it consistently improves the accuracy-efficiency trade-off over training-free baselines, particularly under severe compression. On LLaVA-NeXT-7B, AnchorPrune preserves 97.6% of full-token performance using only 160 of 2,880 visual tokens. These results establish relevance-anchored contextual expansion as an effective principle for efficient multimodal inference. Code is available at https://github.com/MULTI-cau/AnchorPrune.

Related papers

• StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection [17.809800994999225]
  StackTok is a training-free selector that treats query relevance as the objective and visual coverage as budget-calibrated support.<n>It ranks first among training-free selectors in every tested model--budget setting.<n>On high-resolution LLaVA-NeXT-7B, it retains 95.26% of full-token performance with only 160 of 2,880 (5.6%) visual tokens.
  arXiv  Detail & Related papers  (2026-09-15T08:35:57Z)
• Centering before Pruning: Lightweight Geometry Correction for Diversity-Based Visual Token Pruning in LVLMs [7.371189496638082]
  We find that similarities between raw visual tokens are strongly concentrated in the positive range, limiting their ability to distinguish non-redundant tokens.<n>We propose the textbfCentered Geometry textbfPruner (Cen-Prune), which measures subset diversity using centered cosine similarity.<n>Cen-Prune provides robust improvements in overall performance across existing diversity-based pruners.
  arXiv  Detail & Related papers  (2026-08-31T05:13:29Z)
• Combating Textual Noise and Redundancy: Entropy-Aware Dense Visual Token Pruning [16.79729969987944]
  We propose Entropy-Aware Dense Pruning (EADP), a framework that reformulates pruning as a structured compression problem.<n>EADP first leverages statistical entropy to quantify and filter out textual noise, yielding a robust, fine-grained instruction score.
  arXiv  Detail & Related papers  (2026-07-02T17:50:57Z)
• Moving Beyond Diversity: Visual Token Pruning as Subspace Reconstruction for Efficient VLMs [7.371189496638082]
  We introduce SPARE, a subspace reconstruction method that reformulates token pruning as a column subset selection problem.<n>We show that SPARE consistently achieves state-of-the-art performance, with strong gains on tasks.
  arXiv  Detail & Related papers  (2026-06-17T04:45:10Z)
• PARCEL: Pool-Anchored Resampling with Conditioned Elastic Queries for Efficient Vision-Language Understanding [88.17174909130188]
  Large Vision-Language Models (LVLMs) map visual inputs into dense token sequences, imposing a quadratic computational bottleneck for inference.<n>We introduce PARCEL, a visual tokenization architecture that dynamically partitions the labor of feature extraction.<n>We show that PARCEL consistently outperforms existing matryoshka baselines across visual-token budgets while preserving the "train once, deploy anywhere" paradigm.
  arXiv  Detail & Related papers  (2026-05-28T15:57:31Z)
• Evading Visual Aphasia: Contrastive Adaptive Semantic Token Pruning for Vision-Language Models [52.78477729846771]
  We introduce COAST (COntrastive Adaptive Semantic Token Pruning), a training-free pruning framework that casts compression as adaptive semantic routing.<n> COAST consistently outperforms strong pruning baselines across token budgets and generalizes across multiple LVLM families.
  arXiv  Detail & Related papers  (2026-05-10T09:07:04Z)
• D2Pruner: Debiased Importance and Structural Diversity for MLLM Token Pruning [49.16227597771663]
  D2Pruner is a framework that combines debiased importance with a structural pruning mechanism.<n>It reduces FLOPs by 74.2% while retaining 99.2% of its original performance.<n>It marks a significant advancement with up to 63. 53% improvement over existing methods.
  arXiv  Detail & Related papers  (2025-12-22T14:42:31Z)
• Training-free Context-adaptive Attention for Efficient Long Context Modeling [57.703159205740185]
  Training-free Context-adaptive Attention (TCA-Attention) is a training-free sparse attention mechanism that selectively attends to only the informative tokens for efficient long-context inference.<n>TCA-Attention achieves a 2.8$times$ speedup and reduces KV cache by 61% at 128K context length while maintaining performance comparable to full attention.
  arXiv  Detail & Related papers  (2025-12-10T01:54:57Z)
• ZSPAPrune: Zero-Shot Prompt-Aware Token Pruning for Vision-Language Models [7.7352936204066]
  We propose a novel, zero-shot method to model visual token pruning as a balance between task relevance and information diversity.<n>Our method achieves performance that matches or surpasses the state-of-the-art with only minimal accuracy loss.<n>These gains are accompanied by significant reductions in GPU memory footprint and inference latency.
  arXiv  Detail & Related papers  (2025-10-20T06:18:47Z)
• TrimTokenator: Towards Adaptive Visual Token Pruning for Large Multimodal Models [4.779482139419908]
  We introduce a mutual information-based token pruning strategy that removes visual tokens semantically with textual tokens.<n>Our method maintains strong performance while reducing textual tokens by 88.9% on models such as LLaVA-15-7B and LLaVA--7B.
  arXiv  Detail & Related papers  (2025-08-30T02:43:50Z)
• Reinforcing Video Reasoning with Focused Thinking [65.85683941058916]
  We propose TW-GRPO, a novel framework that enhances visual reasoning with focused thinking and dense reward granularity.<n>Specifically, we employ a token weighting mechanism that prioritizes tokens with high informational density.<n>We also reformulate RL training by shifting from single-choice to multi-choice QA tasks.
  arXiv  Detail & Related papers  (2025-05-30T15:42:19Z)
• TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model [56.43860351559185]
  We introduce textbfTopV, a compatible textbfTOken textbfPruning with inference Time Optimization for fast and low-memory textbfVLM.<n>Our framework incorporates a visual-aware cost function to measure the importance of each source visual token, enabling effective pruning of low-importance tokens.
  arXiv  Detail & Related papers  (2025-03-24T01:47:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.