An AI4AI Framework for Visual Token Pruning

• URL: http://arxiv.org/abs/2608.07193v1
• Date: Fri, 07 Aug 2026 13:07:40 GMT
• Title: An AI4AI Framework for Visual Token Pruning
• Abstract summary: We build an AI4AI framework for visual-token pruning.<n>AutoPrune is a training-free framework for visual-token pruning policy design.
• Score: 30.316781118289043
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual-token pruning can substantially reduce the inference cost of multimodal large language models (MLLMs), yet existing methods largely rely on fixed, handcrafted heuristics and costly expert trial and error. As pruning objectives, budgets, and model architectures diversify, manually navigating the expanding design space becomes increasingly difficult. This paper aims to build an AI4AI framework for visual-token pruning by addressing a natural question: Can large language models automatically design effective visual-token reduction algorithms? Although LLMs possess broad algorithmic knowledge and strong reasoning capabilities, translating such general knowledge into effective solutions for a specialized task remains nontrivial. We argue that the key lies in designing an appropriate search-state representation that connects the internal knowledge of LLMs with the structural requirements and constraints of visual-token pruning. Based on this insight, we propose AutoPrune, a training-free framework for LLM-driven visual-token pruning policy design. At its core, AutoPrune introduces a Token Pruning Domain-Specific Language (TPDSL) comprising 131 reusable atoms for budget control, token scoring, selection constraints, and token reassembly. A key property of TPDSL is that it represents each search state as a residual modification of a strong base policy. This residual formulation narrows the search space and directs the LLM's attention toward the policy components that are most consequential for performance. Experiments on 14 multimodal benchmarks and three MLLM backbones demonstrate the effectiveness, efficiency, and transferability of AutoPrune. Even when removing 94.4% of visual tokens, AutoPrune preserves more than 99% of full-token performance while reducing FLOPs by 9.9x and prefill latency by 6.4x.

Related papers

• TOPS: First-Principles Visual Token Pruning via Constructing Token Optimal Preservation Sets for Efficient MLLM Inference [54.41143483594129]
  Multimodal large language models (MLLMs) have achieved strong multimodal reasoning capabilities, but their efficiency is limited by the large number of visual tokens.<n>Visual token pruning offers a natural solution, yet existing methods are imperfect.<n>We propose TOPS, a training-free and model-agnostic pruning module that can be applied to various MLLMs.
  arXiv  Detail & Related papers  (2026-06-25T15:29:37Z)
• QAPruner: Quantization-Aware Vision Token Pruning for Multimodal Large Language Models [18.353831760548267]
  Multimodal Large Language Models (MLLMs) have shown strong reasoning ability, but their high computational and memory costs hinder deployment in resource-constrained settings.<n>While Post-Training Quantization (PTQ) and vision token pruning are standard compression techniques, they are usually treated as independent optimizations.<n>In this paper, we show that these two techniques are strongly coupled: naively applying semantic-based token pruning to PTQ-optimized MLLMs can discard activation outliers that are important for numerical stability and thus worsen quantization errors in low-bit regimes.<n>We propose a quantization-aware vision token pruning framework
  arXiv  Detail & Related papers  (2026-04-03T07:32:07Z)
• ResPrune: Text-Conditioned Subspace Reconstruction for Visual Token Pruning in Large Vision-Language Models [59.94664910790462]
  ResPrune is a training-free visual token pruning framework for large vision-language models.<n>It selects a compact yet informative subset of visual tokens.<n>It achieves effective reductions in computation, memory consumption, and inference latency.
  arXiv  Detail & Related papers  (2026-03-22T07:44:45Z)
• Language-Guided Token Compression with Reinforcement Learning in Large Vision-Language Models [19.87835200940665]
  TPRL learns adaptive pruning trajectories through language-guided sequential optimization tied directly to end-task performance.<n>We formulate visual token pruning as a sequential decision process with explicit state transitions.<n>We employ a self-supervised autoencoder to compress visual tokens into a compact state representation for efficient policy learning.
  arXiv  Detail & Related papers  (2026-03-11T09:52:44Z)
• Informed Routing in LLMs: Smarter Token-Level Computation for Faster Inference [7.690958366125321]
  This paper introduces informed routing, a new paradigm that proactively addresses these issues.<n>We propose the Lightweight Feature Forecaster (LFF), a small predictive module that estimates a unit's output before routing decisions are made.<n>Experiments on both language modeling and reasoning tasks show that informed routing achieves state-of-the-art efficiency-performance trade-offs.
  arXiv  Detail & Related papers  (2025-10-10T09:59:36Z)
• AutoPrune: Each Complexity Deserves a Pruning Policy [58.448785378705566]
  Complexity Pruning (AutoPrune) is a training-free, plug-and-play framework that tailors pruning policies to varying sample and task complexities.<n>We evaluate AutoPrune on standard vision-Adaptive tasks and on Vision-Language-Action models for autonomous driving.
  arXiv  Detail & Related papers  (2025-09-28T15:09:00Z)
• VFlowOpt: A Token Pruning Framework for LMMs with Visual Information Flow-Guided Optimization [70.98122339799218]
  Large Multimodal Models (LMMs) excel in visual-language tasks by leveraging numerous visual tokens for fine-grained visual information.<n>Previous research aimed at reducing visual tokens during inference typically leverages importance maps derived from attention scores among vision-only tokens or vision-language tokens to prune tokens across one or multiple pruning stages.<n>We propose VFlowOpt, a token pruning framework that introduces an importance map derivation process and a progressive pruning module with a recycling mechanism.<n> Experiments demonstrate that VFlowOpt can prune 90% of visual tokens while maintaining comparable performance, leading to an 89% reduction in KV-Cache memory and 3.8
  arXiv  Detail & Related papers  (2025-08-07T09:47:21Z)
• A Glimpse to Compress: Dynamic Visual Token Pruning for Large Vision-Language Models [94.49953824684853]
  We introduce a dynamic pruning framework, GlimpsePrune, inspired by human cognition.<n>It takes a data-driven ''glimpse'' and prunes irrelevant visual tokens in a single forward pass before answer generation.<n>An enhanced GlimpsePrune+ achieves 110% of the baseline performance while maintaining a similarly high pruning rate.
  arXiv  Detail & Related papers  (2025-08-03T02:15:43Z)
• Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters [54.01228554126122]
  Vision Language Models (VLMs) have demonstrated strong capabilities across various visual understanding and reasoning tasks.<n>To reduce inference costs, one can either downsize the Large Language Models (LLMs) or reduce the number of input tokens needed to represent the image.<n>We take the first steps toward designing token compression algorithms tailored for high-compression settings.
  arXiv  Detail & Related papers  (2024-11-05T18:54:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.